DESCRIPTION 



VISUAL PROCESSING DEVICE, VISUAL PROCESSING METHOD, 
VISUAL PROCESSING PROGRAM, AND SEMICONDUCTOR DEVICE 

5 

Background of the Invention 
Technical Field 

The present invention relates to visual processing devices, in 
10 particular to visual processing devices that perform visual processing 
such as spatial processing or gradation processing of an image signal. 

Background Art 

Spatial processing and gradation processing are known as 
15 techniques for visually processing image signals of an original image. 

Spatial processing involves using the pixels around a pixel to be 
filtered when processing that pixel. Further, the technique of using an 
image signal that has been subjected to spatial processing to perform 
contrast enhancement or dynamic range (DR) compression, for example, 
20 of an original image is known. With contrast enhancement, the 
difference between the original image and the blur signal (the sharp 
component of the image) is added to the original image, sharpening the 
image. With DR compression, a portion of the blur signal is subtracted 
from the original image, compressing the dynamic range. 
25 Gradation processing is processing in which a lookup table (LUT) 

is used to transform a pixel value for each pixel being processed without 
regard for the pixels around the pixel being processed, and is also 
referred to as "gamma correction." For example, in the case of contrast 
enhancement, transformation of the pixel value is performed using a 
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LUT that produces a gradation of gray levels that appear frequently 
(whose area is large) in the original image. Well-known examples of 
gradation processing using a LUT include gradation processing in which 
a single LUT is chosen and used for the entire original image (histogram 
5 equalization) and gradation processing in which the original image is 
partitioned into a plurality of image regions and a LUT is chosen and 
used for each image region (local histogram equalization) (for example, 
see JP 2000-57335A (pg. 3, Figs. 13 to 16)). 

Visual processing that combines spatial processing and gradation 
10 processing also is known. Conventional visual processing that combines 
spatial processing and gradation processing is described below using Fig. 
48 and Fig. 51. 

Fig. 48 shows a visual processing device 400 that performs edge 
enhancement and contrast enhancement utilizing unsharp masking. 

15 The visual processing device 400 shown in Fig. 48 is provided with a 
spatial processing portion 401 that performs spatial processing with 
respect to the input signal IS and outputs an unsharp signal US, a 
subtracting portion 402 that subtracts the unsharp signal US from the 
input signal IS and outputs a difference signal DS, an enhancing portion 

20 403 that performs enhancement of the difference signal DS and outputs 
an enhanced signal TS, and a summing portion 404 that takes the sum of 
the input signal IS and the enhanced signal TS and outputs an output 
signal OS. 

Here, enhancement processing is performed with respect to the 
25 difference signal DS using a linear or nonTinear enhancement function. 
Fig. 49 shows the enhancement functions Rl to R3. The horizontal axis 
in Fig. 49 marks the difference signal DS and the vertical axis marks the 
enhanced signal TS. The enhancement function Rl is an enhancement 
function that is linear with respect to the difference signal DS. The 
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enhancement function Rl is a gain adjustment function expressed for 
example by Rl(x) = 0.5x (where x is the value of the difference signal DS). 
The enhancement function R2 is a nonTinear enhancement function with 
respect to the difference signal DS, and is a function that inhibits 
5 extreme contrasts. In other words, a greater inhibitory effect (an 
inhibitory effect due to a larger inhibition rate) is exhibited with respect 
to an input x having a large absolute value (where x is the value of the 
difference signal DS). For example, the enhancement function R2 is 
expressed by a graph having a smaller slope the larger the absolute 

10 value of the input x. The enhancement function R3 is a non-linear 
enhancement function for the difference signal DS that inhibits the noise 
component in small amplitudes. That is, a greater inhibitory effect (an 
inhibitory effect due to a larger inhibition rate) is attained with respect 
to an input x having a small absolute value (where x is the value of the 

15 difference signal DS). For example, the enhancement function R3 is 
expressed by a graph having a larger slope the greater the absolute 
value of the input x. The enhancing portion 403 can use any of these 
enhancement functions Rl to R3. 

The difference signal DS is the sharp component of the input 

20 signal IS. With the visual processing device 400, the intensity of the 
difference signal DS is transformed and added to the input signal IS. 
Thus, the output signal OS is the input signal IS in which the edges and 
the contrast have been enhanced. 

Fig. 50 shows a visual processing device 406 that improves the 

25 local contrast (intensity) (for example, see Japanese Patent JP 2832954 
(pg. 2, Fig. 5)). The visual processing device 406 shown in Fig. 50 is 
provided with a spatial processing portion 407, a subtracting portion 408, 
a first transformation portion 409, a multiplying portion 410, a second 
transformation portion 411, and a summing portion 412. The spatial 
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processing portion 407 performs spatial processing with respect to the 
input signal IS and outputs an unsharp signal US. The subtracting 
portion 408 subtracts the unsharp signal US from the input signal IS 
and outputs a difference signal DS. The first transformation portion 
5 409 outputs an amplification coefficient signal GS for locally amplifying 
the difference signal DS based on the intensity of the unsharp signal US. 
The multiplying portion 410 takes the product of the difference signal DS 
and the amplification coefficient signal GS and outputs a contrast 
enhanced signal HS in which the difference signal DS has been locally 
10 amplified. The second transformation portion 411 locally corrects the 
intensity of the unsharp signal US and outputs a corrected unsharp 
signal AS. The summing portion 412 takes the sum of the contrast 
enhanced signal HS and the corrected unsharp signal AS and outputs an 
output signal OS. 

15 The amplification coefficient signal GS is a non-linear weight 

coefficient that locally corrects the contrast in portions of the input 
signal IS where the contrast is unsuitable. For this reason, portions of 
the input signal IS having suitable contrast are output unchanged, and 
those portions with an unsuitable contrast are corrected and then 

20 output. 

Fig. 51 shows a visual processing device 416 that performs 
compression of the dynamic range (for example, see JP 2001-298619A (pg. 
3, Fig. 9)). The visual processing device 416 shown in Fig. 51 is 
provided with a spatial processing portion 417 that performs spatial 
25 processing with respect to the input signal IS and outputs an unsharp 
signal US, a LUT computation portion 418 that uses a LUT to perform 
an inverse transformation of the unsharp signal US to produce a LUT 
processed signal LS which it then outputs, and a summing portion 419 
that takes the sum of the input signal IS and the LUT processed signal 
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LS and outputs an output signal OS. 

The LUT processed signal LS is added to the input signal IS to 
compress the dynamic range of low-frequency components of the input 
signal IS (frequency components lower than the cutoff frequency of the 
5 spatial processing portion 417). As a result, the dynamic range of the 
input signal IS is compressed while its high-frequency components are 
retained. 

Summary of the Invention 

10 In the above visual processing, there is a need to obtain an image 

that has a better visual effect for the person who will view the visually 
processed image. For example, if the visually processed image is to be 
displayed, then the display environment will have an effect on how the 
image that is displayed will be viewed. 

15 Accordingly, a goal of the present invention is to obtain an image 

that has a better visual effect for the person that will view the visually 
processed image. 

The visual processing device according to a first aspect of the 
invention comprises parameter output means and transformation means. 

20 The parameter output means outputs a luminance adjustment 
parameter based on a parameter expressing the ambient light. The 
transformation means transforms a luminance of a target pixel based on 
the luminance adjustment parameter that is output by the parameter 
output means, a luminance of a target pixel to be subjected to visual 

25 processing, and a luminance of surrounding pixels positioned in an area 
around the target pixel. 

The parameter expressing the ambient light is for example 
measured by an optical sensor that senses the intensity of the light and 
input to the parameter output means. Alternatively, the parameter 
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expressing the ambient light can be created at the discretion of the user 
and then input to the parameter output means. 

The luminance adjustment parameter is for example a lookup 
table (LUT) that stores the target pixel luminance after transformation 
5 with respect to the luminance of the target pixel, the luminance of 
surrounding pixels, or the result of a computation involving these values, 
or coefficient matrix data for transforming the luminance of the target 
pixel, the luminance of surrounding pixels, or the result of a computation 
involving these values. The luminance adjustment parameter can also 

10 include the parameter expressing the ambient light. 

The parameter output means for example selects the luminance 
adjustment parameter corresponding to the parameter that expresses 
the ambient light from among a plurality of luminance adjustment 
parameters and outputs this, or creates a luminance adjustment 

15 parameter based on a computation that incorporates the parameter 
expressing the ambient light and outputs this. 

With the visual processing device according to this aspect of the 
invention, it is possible to achieve visual processing that is in 
correspondence with the ambient light. That is, it is possible to obtain 

20 an image that has a better visual effect for the person who will view the 
visually processed image. 

The visual processing device according to a second aspect of the 
invention is the visual processing device according to the first aspect of 
the invention in which the parameter output means outputs the 

25 luminance adjustment parameter based on the parameter expressing the 
ambient light and an outside parameter that is input from an outside 
portion. 

The outside parameter is a parameter that expresses a visual 
effect that is desired by the user who will view the image. More 



specifically, it is a value such as the contrast desired by the user who will 
view the image (the same applies in the following description). 

The parameter output means outputs the luminance adjustment 
parameter based on the parameter expressing the ambient light and the 
5 outside parameter. Here, the luminance adjustment parameter can for 
example also include the outside parameter. 

With the visual processing device according to this aspect of the 
invention, it is possible to achieve visual processing that corresponds to 
the ambient light and the outside parameter. For example, the value of 

10 the contrast desired by the user who will view the image can be set by 
the user himself and thus an image that has a better visual effect from 
the perspective of the user can be obtained. 

The visual processing device according to a third aspect of the 
invention is the visual processing device according to the first aspect of 

15 the invention in which the operation of the parameter output means is 
switched according to a switch signal, between a first mode of outputting 
the luminance adjustment parameter based on the parameter expressing 
the ambient light and a second mode of outputting the luminance 
adjustment parameter based on the parameter expressing the ambient 

20 light and an outside parameter that is input from an outside portion. 

In the first mode, a luminance adjustment parameter that 
corresponds to the ambient light is output. In the second mode, a 
luminance adjustment parameter that corresponds to the ambient light 
and an outside parameter is output. 

25 For example, in the first mode a predetermined luminance 

adjustment parameter that has been set in the system in advance is 
output. Also, for example in the second mode the user himself sets a 
value of the contrast, for example, at which he would like to view the 
image, and a luminance adjustment parameter that correspondence to 
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this set value and the ambient light is output. 

With the visual processing device according to this aspect of the 
invention, it is possible to switch between using a value of the contrast, 
for example, that has been set by the user who will view the image, and 
5 using a predetermined value that has been set in the system in advance. 

The visual processing device according to a fourth aspect of the 
invention is the visual processing device according to the first aspect of 
the invention in which the transformation means performs a 
computation for enhancing a difference or a ratio between the luminance 
10 of the target pixel and the luminance of the surrounding pixel. 

Here, the computation for enhancing includes not only 
enhancement in the positive direction but also enhancement in the 
negative direction. For example, it is inclusive of processing for 
smoothing the luminance of the target pixel and the luminance of the 
15 surrounding pixels, or processing for enhancing local contrast. 

With the visual processing device according to this aspect of the 
invention, it is for example possible to enhance local contrast to maintain 
the viewed contrast in an environment where ambient light is present. 

The visual processing device according to a fifth aspect of the 
20 invention is the visual processing device according tothe first aspect of 
the invention, further comprising a time change adjustment portion for 
controlling the change over time in the parameter expressing the 
ambient light or the luminance adjustment parameter. 

Here, the time change adjustment portion controls the change 
25 over time by easing the response to the change in the parameter over 
time, or delaying the response to the change in the parameter over time. 
In the case of easing the response, the time change adjustment portion 
can be constituted by a smoothing filter such as an IIR filter or means 
that outputs a value obtained by integrating the values of the respective 



parameters or a value obtained by taking a mean of the integrated 
values. 

With the visual processing device according to this aspect of the 
invention, by controlling the change over time in the parameter 
5 expressing the ambient light or the luminance adjustment parameter, it 
is possible to inhibit abrupt parameter fluctuations, and thus allows 
flickering in the displayed image to be inhibited. 

The visual processing method according to a sixth aspect of the 
invention comprises a parameter output step and a transformation step. 
10 The parameter output step is a step of outputting a luminance 
adjustment parameter based on a parameter expressing the ambient 
light. The transformation step is a step of transforming a luminance of 
a target pixel based on the luminance adjustment parameter that is 
output through the parameter output step, a luminance of a target pixel 
15 to be subjected to visual processing, and a luminance of surrounding 
pixels positioned in an area around the target pixel. 

The parameter expressing the ambient light is for example 
measured by an optical sensor that senses the intensity of the light. 
Alternatively, the parameter expressing the ambient light can be created 
20 at the discretion of the user. 

The luminance adjustment parameter is for example a lookup 
table (LUT) that stores the target pixel luminance after transformation 
with respect to the luminance of the target pixel, the luminance of 
surrounding pixels, or the result of a computation involving these values, 
25 or coefficient matrix data for transforming the luminance of the target 
pixel, the luminance of surrounding pixels, or the result of a computation 
involving these values. The luminance adjustment parameter can also 
include the parameter expressing the ambient light. 

The parameter output step is for example a step of selecting and 
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outputting the luminance adjustment parameter corresponding to the 
parameter that expresses the ambient light from among a plurality of 
luminance adjustment parameters, or of creating a luminance 
adjustment parameter based on a computation that incorporates the 
5 parameter expressing the ambient light and outputting this. 

With the visual processing method according to this aspect of the 
invention, it is possible to achieve visual processing that corresponds to 
the ambient light. That is, it is possible to perform visual processing 
that yields a better visual effect. 

10 The visual processing program according to a seventh aspect of 

the invention is a program that causes a computer to perform a visual 
processing method. The visual processing method comprises a 
parameter output step and a transformation step. The parameter 
output step is a step of outputting a luminance adjustment parameter 

15 based on a parameter expressing the ambient light. The transformation 
step is a step of transforming a luminance of a target pixel based on the 
luminance adjustment parameter that is output through the parameter 
output step, a luminance of a target pixel to be subjected to visual 
processing, and a luminance of surrounding pixels positioned in an area 

20 around the target pixel. 

The parameter expressing the ambient light is for example 
measured by an optical sensor that senses the intensity of the light. 
Alternatively, the parameter expressing the ambient light can be created 
at the discretion of the user. 

25 The luminance adjustment parameter is for example a lookup 

table (LUT) that stores the target pixel luminance after transformation 
with respect to the luminance of the target pixel, the luminance of 
surrounding pixels, or the result of a computation involving these values, 
or coefficient matrix data for transforming the luminance of the target 
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pixel, the luminance of surrounding pixels, or the result of a computation 
involving these values. The luminance adjustment parameter can also 
include the parameter expressing the ambient light. 

The parameter output step is for example a step of selecting and 
5 outputting the luminance adjustment parameter corresponding to the 
parameter that expresses the ambient light from among a plurality of 
luminance adjustment parameters, or of creating a luminance 
adjustment parameter based on a computation that incorporates the 
parameter expressing the ambient light and outputting this. 

10 With the visual processing program according to this aspect of the 

invention, it is possible to achieve visual processing that is in 
correspondence with the ambient light. That is, it is possible to perform 
visual processing that yields a better visual effect. 

The semiconductor device according to an eighth aspect of the 

15 invention comprises a parameter output portion and a transformation 
portion. The parameter output portion outputs a luminance adjustment 
parameter based on a parameter expressing the ambient light. The 
transformation portion transforms a luminance of a target pixel based on 
the luminance adjustment parameter that is output by the parameter 

20 output portion, a luminance of the target pixel to be subjected to visual 
processing, and a luminance of surrounding pixels positioned in an area 
around the target pixel. 

The parameter expressing the ambient light is for example 
measured by an optical sensor that senses the intensity of the light and 

25 input to the parameter output portion. Alternatively, the parameter 
expressing the ambient light can be created at the discretion of the user 
and then input to the parameter output portion. 

The luminance adjustment parameter is for example a lookup 
table (LUT) that stores the target pixel luminance after transformation 
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with respect to the luminance of the target pixel, the luminance of 
surrounding pixels, or the result of a computation involving these values, 
or coefficient matrix data for transforming the luminance of the target 
pixel, the luminance of surrounding pixels, or the result of a computation 
5 involving these values. The luminance adjustment parameter can also 
include the parameter expressing the ambient light. 

The parameter output portion for example selects and then 
outputs a luminance adjustment parameter corresponding to the 
parameter that expresses the ambient light from among a plurality of 
10 luminance adjustment parameters, or creates a luminance adjustment 
parameter based on a computation that incorporates the parameter 
expressing the ambient light and outputs this. 

With the semiconductor device according to this aspect of the 
invention, it is possible to achieve visual processing that is in 
15 correspondence with the ambient light. That is, it is possible to perform 
visual processing that yields a better visual effect. 

With the visual processing device according to the present 
invention, it is possible to obtain an image that has a better visual effect 
for the user who will view the visually processed image. 

20 

Brief Description of Drawings 

Fig. 1 is a block diagram describing the structure of the visual 
processing device 1 (first embodiment). 

Fig. 2 is an example of the profile data (first embodiment). 
25 Fig. 3 is a flowchart for describing the visual processing method 

(first embodiment). 

Fig. 4 is a block diagram describing the structure of the visual 
processing portion 500 (first embodiment). 

Fig. 5 is an example of the profile data (first embodiment). 
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Fig. 6 is a block diagram describing the structure of the visual 
processing device 520 (first embodiment). 

Fig. 7 is a block diagram describing the structure of the visual 
processing device 525 (first embodiment). 
5 Fig. 8 is a block diagram describing the structure of the visual 

processing device 530 (first embodiment). 

Fig. 9 is a block diagram describing the structure of the profile 
data registration device 701 (first embodiment). 

Fig. 10 is a flowchart explaining the visual processing profile 
10 creation method (first embodiment). 

Fig. 11 is a block diagram describing the structure of the visual 
processing device 901 (first embodiment). 

Fig. 12 is a graph showing the relationship between the input 
signal IS' and the output signal OS' after changing the extent change 
15 function fk(z) (first embodiment). 

Fig. 13 is a graph showing the extent change functions fl(z) and 
f2(z) (first embodiment). 

Fig. 14 is a block diagram describing the structure of the visual 
processing device 905 (first embodiment). 
20 Fig. 15 is a block diagram describing the structure of the visual 

processing device 11 (first embodiment). 

Fig. 16 is a block diagram describing the structure of the visual 
processing device 21 (first embodiment). 

Fig. 17 is an explanatory diagram describing the dynamic range 
25 compression function F4 (first embodiment). 

Fig. 18 is an explanatory diagram describing the enhancement 
function F5 (first embodiment). 

Fig. 19 is a block diagram describing the structure of the visual 
processing device 31 (first embodiment). 



13 



Fig. 20 is a block diagram describing the structure of the visual 
processing device 41 (first embodiment). 

Fig. 21 is a block diagram describing the structure of the visual 
processing device 51 (first embodiment). 
5 Fig. 22 is a block diagram describing the structure of the visual 

processing device 61 (first embodiment). 

Fig. 23 is a block diagram describing the structure of the visual 
processing device 71 (first embodiment). 

Fig. 24 is a block diagram describing the structure of the visual 
10 processing device 600 (second embodiment). 

Fig. 25 is a graph describing the transformation by Eq. M20 
(second embodiment). 

Fig. 26 is a graph describing the transformation by Eq. M2 
(second embodiment). 
15 Fig. 27 is a graph describing the transformation by Eq. M21 

(second embodiment). 

Fig. 28 is a flowchart describing the visual processing method 
(second embodiment). 

Fig. 29 is a graph showing the trend of the function al(A) (second 
20 embodiment). 

Fig. 30 is a graph showing the trend of the function a2(A) (second 
embodiment). 

Fig. 31 is a graph showing the trend of the function a3(A) (second 
embodiment). 

25 Fig. 32 is a graph showing the trend of the function a4(A,B) 

(second embodiment). 

Fig. 33 is a block diagram describing the structure of the actual 
contrast setting portion 605 as a modified example (second embodiment). 
Fig. 34 is a block diagram describing the structure of the actual 
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contrast setting portion 605 as a modified example (second embodiment). 

Fig. 35 is a flowchart describing the operation of the control 
portion 605e (second embodiment). 

Fig. 36 is a block diagram describing the structure of the visual 
5 processing device 600 provided with the color difference correcting 
portion 608 (second embodiment). 

Fig. 37 is an explanatory diagram describing an overview of color 
difference correction (second embodiment). 

Fig. 38 is a flowchart describing the estimation computation in 
10 the color difference correcting portion 608 (second embodiment). 

Fig. 39 is a block diagram describing the structure of the visual 
processing device 600 as a modified example (second embodiment). 

Fig. 40 is a block diagram describing the structure of the visual 
processing device 910 (third embodiment). 
15 Fig. 41 is a block diagram describing the structure of the visual 

processing device 920 (third embodiment). 

Fig. 42 is a block diagram describing the structure of the visual 
processing device 920' (third embodiment). 

Fig. 43 is a block diagram describing the structure of the visual 
20 processing device 920" (third embodiment). 

Fig. 44 is a block diagram describing the overall configuration of 
the content supply system (fourth embodiment). 

Fig. 45 is an example of a portable telephone furnished with the 
visual processing device of the present invention (fourth embodiment). 
25 Fig. 46 is a block diagram describing the configuration of the 

portable telephone (fourth embodiment). 

Fig. 47 is an example of the system for digital broadcast (fourth 
embodiment). 

Fig. 48 is a block diagram describing the structure of the visual 
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processing device 400, which employs unsharp masking (background 
art). 

Fig. 49 is an explanatory diagram describing the enhancement 
functions Rl to R3 (background art). 
5 Fig. 50 is a block diagram describing the structure of the visual 

processing device 406, which performs an improvement of the local 
contrast (background art). 

Fig. 51 is a block diagram describing the structure of the visual 
processing device 416, which performs a compression of the dynamic 
10 range (background art). 

Detailed Description of the Invention 

Hereinafter, first through fourth embodiments serving as best 
modes for implementing the present invention will be described. 
15 The first embodiment describes a visual processing device that 

employs a two-dimensional LUT. 

The second embodiment describes a visual processing device that 
performs correction of the ambient light when ambient light is present in 
the environment in which the image is to be displayed. 
20 The third embodiment describes applied examples of the first 

embodiment and the second embodiment. 

The fourth embodiment describes applied examples of the first 
through third embodiments. 

25 [First Embodiment] 

A visual processing device 1 that employs a two-dimensional LUT 
is described below as a first embodiment of the present invention using 
Figs. 1 to 10. A modified example of this visual processing device is 
described using Figs. 11 to 14. Further, a visual processing device that 
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achieves visual processing that is equivalent to that of the visual 
processing device 1 is described using Figs. 15 to 23. 

The visual processing device 1 is a device for performing visual 
processing such as spatial processing and gradation processing of an 
5 image signal. The visual processing device I, together with a device 
that performs color processing of an image signal, constitutes an image 
processing device in a device that handles images, such as a computer, a 
television, a digital camera, a portable telephone, a PDA, a printer, or a 
scanner. 

10 <Visual Processing Device 1> 

Fig. 1 shows the basic structure of the visual processing device 1, 
which performs visual processing of an image signal (input signal IS) 
and outputs the result as a visually processed image (output signal OS). 
The visual processing device 1 is provided with a spatial processing 

15 portion 2 that performs spatial processing of the luminance value of each 
pixel of an original image that has been obtained as an input signal IS 
and outputs the result as an unsharp signal US, and a visual processing 
portion 3 that performs visual processing of the original image using the 
input signal IS and the unsharp signal US for the same pixel and 

20 outputs the result as an output signal OS. 

The spatial processing portion 2 for example obtains the unsharp 
signal US with a low-pass spatial filter that permits the passage of only 
the low-frequency space of the input signal IS. As the low-pass spatial 
filter it is possible to use a FIR (Finite Impulse Response) -type low-pass 

25 spatial filter or an IIR (Infinite Impulse Response) -type low-pass spatial 
filter, which are commonly used to create unsharp signals. 

The visual processing portion 3 has a two-dimensional LUT 4 
that lists the relationship between the input signal IS and the unsharp 
signal US, and the output signal OS, and references the two-dimensional 
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LUT 4 with the input signal IS and the unsharp signal US and outputs 
an output signal OS. 

<Two-Dimensional LUT 4> 

Matrix data referred to as profile data are registered to the 
5 two-dimensional LUT 4. The profile data has a row (or column) for each 
pixel value of the input signal IS and a column (or row) for each pixel 
value of the unsharp signal US, and the pixel values of the output signal 
OS that correspond to the combination of the input signal IS and the 
unsharp signal US are stored as elements of the rows and columns. The 

10 profile data are registered to the two-dimensional LUT 4 by a profile 
data registration device 8 that is provided in or connected to the visual 
processing device 1. The profile data registration device 8 stores data of 
a plurality of profiles that have been created in advance by a personal 
computer (PC) or the like. For example, it stores data of a plurality of 

15 profiles for achieving contrast enhancement, D-range compression, or 
gradation correction, for example (for a more detailed description, see 
the section <Profile Data> below). Thus, with the visual processing 
device 1, the profile data registration device 8 can be used to change the 
profile data registered to the two-dimensional LUT 4, and this allows a 

20 variety of types of visual processing to be achieved. 

An example of the profile data is shown in Fig. 2. The profile 
data shown in Fig. 2 causes the visual processing device 1 to execute 
processing equivalent to that of the visual processing device 400 shown 
in Fig. 48. In Fig. 2, the profile data takes the form of a 64 x 64 matrix 

25 in which the values of the upper six bits of the luminance values of the 
8-bit input signal IS are shown in the column direction (vertical 
direction) and the value of the upper six bits of the luminance value of 
the 8"bit unsharp signal US are shown in the row direction (horizontal 
direction). The value of the output signal OS is expressed in 8 bits 
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value as the matrix element corresponding to the two luminance values. 

The value C of each element of the profile data shown in Fig. 2 
(the value of the output signal OS) is expressed as C = A + 0.5*(A-B) 
(hereinafter, this is referred to as Equation Mil), using the value A of 
5 the input signal IS (which is a value obtained by discarding the lower 
two bits of the 8 bit input signal IS) and the value B of the unsharp 
signal US (which is a value that is obtained by discarding the lower two 
bits of the 8-bit unsharp signal US). In other words, it can be seen that 
the visual processing device 1 performs processing equivalent to 

10 processing by the visual processing device 400 (see Fig. 48) using the 
enhancement function Rl (see Fig. 49). 

It should be noted that depending on the combination of the value 
A of the input signal IS and the value B of the unsharp signal US, the 
value C obtained from Eq. Mil may be a negative value. In this case, 

15 the element of the profile data corresponding to the value A of the input 
signal IS and the value B of the unsharp signal US may be set to the 
value 0. Further, depending on the combination of the value A of the 
input signal IS and the value B of the unsharp signal US, the value C 
obtained from Eq. Mil may be saturated. That is, it may exceed the 

20 maximum value of 255 that can be expressed with 8 bits. In this case, 
the element of the profile data corresponding to the value A of the input 
signal IS and the value B of the unsharp signal US may be set to the 
value 255. In Fig. 2, the elements of the profile data found in this 
manner are shown by contour line. 

25 When using profile data in which the value C of the elements is 

expressed as C = R6(B) + R5(B) * (A-B) (hereinafter, this is referred to as 
Equation M12), processing equal to that of the visual processing device 
406 shown in Fig. 50 can be achieved. Here, the function R5 is that of 
the first transformation portion 409 outputting an amplification 
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coefficient signal GS from the unsharp signal US, and the function R6 is 
that of the second transformation portion 411 outputting a corrected 
unsharp signal AS from the unsharp signal US. 

When using profile data in which the values C of the elements are 
5 expressed by C = A + R8(B) (hereinafter, this is referred to as Equation 
M13), processing equivalent to that of the visual processing device 416 
shown in Fig. 51 can be achieved. Here, the function R8 is that of 
outputting a LUT processed signal LS from the unsharp signal US. 

It should be noted that if the value C of a particular element of 
10 the profile data found through Eq. M12 or Eq. M13 is outside the range 
0<C<255, then the value C of that element can be regarded as 0 or 255. 

<Visual Processing Method and Visual Processing Program> 

Fig. 3 shows a flowchart that describes the visual processing 
method of the visual processing device 1. The visual processing method 
15 shown in Fig. 3 is a method for performing visual processing of an input 
signal IS (see Fig. l), and is achieved by the hardware in the visual 
processing device 1. 

With the visual processing method shown in Fig. 3, an input 
signal IS is spatially processed by the low-pass spatial filter (step Sll) to 
20 obtain an unsharp signal US. The value of the two-dimensional LUT 4 
that corresponds to the input signal IS and the unsharp signal US is 
looked up and this value is output as the output signal OS (step S12). 
The above processing is performed for each pixel that is received as an 
input signal IS. 

25 It should be noted that each step of the visual processing method 

shown in Fig. 3 can also be achieved on a computer, for example, as a 
visual processing program. 
<Effects> 
(1) 
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When visual processing is performed based on only the value A of 
the input signal IS (for example, in the case of performing a 
transformation based on a one-dimensional gradation transformation 
curve), if pixels having the same darkness are present at different areas 
5 in the image, then the two are transformed to the same brightness. 
More specifically, when brightening a dark area in the background of a 
person in an image, hair of that person having the same darkness will 
also be brightened. 

In contrast to this, the visual processing device 1 performs visual 

10 processing using profile data that have created based on a 
two-dimensional function and correspond to the value A of an input 
signal IS and the value B of an unsharp signal US. For this reason, 
pixels having the same darkness that are present at different areas 
within the image are not transformed uniformly but instead can be made 

15 brighter or darker taking into account information on their surroundings, 
and this allows each region of the image to be adjusted to an ideal 
brightness. More specifically, a background that has the same darkness 
as the hair of a person in the image can be brightened without changing 
the darkness of the hair. 

20 (2) 

With the visual processing device 1, visual processing of the input 
signal IS is performed using the two-dimensional LUT 4. The visual 
processing device 1 has a hardware configuration that is independent of 
the visual processing effect that is to be achieved. That is, the visual 
25 processing device 1 can be constituted by generalized hardware, and this 
is useful for reducing hardware costs, for example. 

(3) 

The profile data that are registered to the two-dimensional LUT 4 
can be changed by the profile data registration device 8. Thus, by 
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changing the profile data it is possible for the visual processing device 1 
to achieve various types of visual processing, without changing the 
hardware configuration of the visual processing device I. More 
specifically, the visual processing device 1 is capable of simultaneously 
5 executing spatial processing and gradation processing. 
(4) 

The profile data that are registered to the two-dimensional LUT 4 
can be calculated in advance. For profile data that have been created 
already, the amount of time required for visual processing using those 

10 data is the same regardless of the complexity of the processing. Thus, it 
is possible to increase the speed of visual processing by using the visual 
processing device 1 because the processing time does not depend on the 
complexity of the visual processing, even if the visual processing requires 
a complex structure when configured by hardware or software. 

15 <Modified Examples> 

(1) 

Fig. 2 shows profile data in the form of a 64 x 64 matrix. Here, 
the effect of the present invention does not depend on the size of the 
profile data. For example, the two-dimensional LUT 4 can also have 
20 profile data that correspond to all the combinations of values possible 
from the input signal IS and the unsharp signal US. For example, if the 
input signal IS and the unsharp signal US are expressed with 8 bits, 
then the profile data can take the form of a 256 x 256 matrix. 

In this case, although the memory capacity required for the 
25 two-dimensional LUT 4 increases, more accurate visual processing can 
be achieved. 

(2) 

In Fig. 2, the profile data have been explained as storing the 
values of the output signals OS for the values of the upper six bits of the 
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luminance value of the 8-bit input signal IS and the values of the upper 
six bits of the luminance value of the 8-bit unsharp signal US. Here, it 
is also possible that the visual processing device 1 is further provided 
with an interpolation portion that performs linear interpolation of the 
5 value of the output signal OS based on the adjacent elements of the 
profile data and the size of the lower two bits of the input signal IS and 
the unsharp signal US. 

In this case, more accurate visual processing can be achieved 
without increasing the memory capacity necessary for the 

10 two-dimensional LUT 4. 

The interpolating portion may be provided in the visual 
processing portion 3 and may output as an output signal OS a value that 
is obtained by linearly interpolating the values stored in the 
two-dimensional LUT 4. 

15 Fig. 4 shows a visual processing portion 500 provided with an 

interpolating portion 501, as a modified example of the visual processing 
portion 3. The visual processing portion 500 is provided with a 
two-dimensional LUT 4 that lists the relationship between an input 
signal IS and an unsharp signal US and a pre-interpolation output 

20 signal NS, and the interpolating portion 501, which receives the 
pre-interpolation output signal NS, the input signal IS and the unsharp 
signal US and outputs an output signal OS. 

The two-dimensional LUT 4 stores the values of the 
pre-interpolation output signal NS for the values of the upper six bits of 

25 the luminance of the 8-bit input signal IS and the values of the upper six 
bits of the luminance of the 8-bit unsharp signal US. The values of the 
pre-interpolation output signal NS are for example stored as 8-bit values. 
When the two-dimensional LUT 4 receives the 8-bit value of the input 
signal IS and the 8-bit value of the unsharp signal US, it outputs the 
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values of the four pre-interpolation output signals NS corresponding to 
the section including each of the values. The section including each of 
the values is the section surrounded by the four pre-interpolation output 
signals NS stored for the combinations of (the value of the upper six bits 
5 of the input signal IS, the value of the upper six bits of the unsharp 
signal US), (the smallest six-bit value that exceeds the value of the 
upper six bits of the input signal IS, the value of the upper six bits of the 
unsharp signal US), (the value of the upper six bits of the input signal IS, 
the smallest six-bit value that exceeds the value of the upper six bits of 
10 the unsharp signal US), and (the smallest six-bit value that exceeds the 
value of the upper six bits of the input signal IS, the smallest six-bit 
value that exceeds the value of the upper six bits of the unsharp signal 
US). 

The value of the lower two bits of the input signal IS and the 
15 value of the lower two bits of the unsharp signal US are input to the 
interpolating portion 501, and using these values, the interpolating 
portion 501 performs linear interpolation of the values of the four 
pre-interpolation output signals NS output by the two-dimensional LUT 
4. More specifically, the interpolating portion 501 uses the value of the 
20 lower two bits of the input signal IS and the value of the lower two bits 
of the unsharp signal US to calculate the weighted mean of the values of 
the four pre-interpolation output signals NS, and outputs this as the 
output signal OS. 

Through the above, more accurate visual processing can be 
25 achieved without increasing the memory capacity required for the 
two-dimensional LUT 4. 

It should be noted that it is also possible for the interpolating 
portion 501 to perform linear interpolation of only one of either the input 
signal IS or the unsharp signal US. 
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(3) 

With the spatial processing performed by the spatial processing 
portion 2, as the unsharp signal US for the input signal IS of a pixel 
being processed it is also possible to output the mean value (simple mean 
5 or weighted mean), the maximum value, the minimum value, or the 
median value of the input signal IS of the pixel being processed and the 
pixels surrounding the pixel being processed. It is further possible to 
output the mean value (simple mean or weighted mean), the maximum 
value, the minimum value, or the median value of only the pixels 
10 surrounding the pixel being processed as the unsharp signal US. 

(4) 

In Fig. 2, the values C of the elements of the profile data are 
created based on the linear function Mil performed with respect to each 
of the value A of the input signal IS and the value B of the unsharp 
15 signal US. However, it is also possible to create the values C of the 
elements of the profile data based on a non-linear function with respect 
to the value A of the input signal IS. 

In this case, it is possible to achieve visual processing that 
corresponds to the viewing properties or visual processing that is suited 
20 for the non-linear characteristics of a device that handles images, such 
as a computer, a television, a digital camera, a portable telephone, a PDA, 
a printer, or a scanner, that outputs an output signal OS. 

It is also possible to create the value C of the each element of the 
profile data based on a non-linear function of the value A of the input 
25 signal IS and the value B of the unsharp signal US, that is, a 
two-dimensional non-linear function. 

For example, when visual processing is performed based on only 
the value A of the input signal IS (for example, in the case of performing 
a transformation based on a one-dimensional gradation transformation 
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curve), if pixels having the same darkness are present at different areas 
in the image, then the two are transformed to the same brightness. 
More specifically, when brightening a dark area in the background of a 
person in an image, hair of that person having the same darkness will 
5 also be brightened. 

On the other hand, if visual processing is performed using profile 
data that have been created based on a two-dimensional non-linear 
function, pixels having the same darkness that are present at different 
areas within the image are not transformed uniformly but instead can be 

10 made brighter or darker taking into account surrounding information, 
and this allows each region of the image to be adjusted to an ideal 
brightness. More specifically, a background that has the same darkness 
as the hair of a person in the image can be brightened without changing 
the darkness of the hair. Moreover, it is also possible to perform visual 

15 processing in which the gradation is maintained, even for those image 
regions in which the pixel values become saturated after visual 
processing according to a linear function. 

Fig. 5 shows an example of such profile data. The profile data 
shown in Fig. 5 are profile data with which the visual processing device 1 

20 can perform contrast enhancement that is suited for the visual 
characteristics. In Fig. 5, the profile data are expressed as a 64 x 64 
matrix, in which the value of the upper six bits of the luminance value of 
the 8-bit input signal IS is shown in the column direction (vertical 
direction) and the value of the upper six bits of the luminance value of 

25 the 8-bit unsharp signal US is shown in the row direction (horizontal 
direction). The output signal OS is expressed as an 8-bit value as the 
matrix element corresponding to the two luminance values. 

The values C of the elements of the profile data shown in Fig. 5 
(the values of the output signal OS) can be expressed by C = 
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F2(F1(A)+F3(F1(A)-F1(B))) (hereinafter, this is referred to as Equation 
M14), using the value A of the input signal IS (which is a value obtained 
by discarding the lower two bits of the 8 bit input signal IS, for example), 
the value B of the unsharp signal US (which is a value that is obtained 
5 by discarding the lower two bits of the 8-bit unsharp signal US, for 
example), a transformation function Fl, an inverse transformation 
function F2 of that transformation function, and an enhancement 
function F3. Here, the transformation function Fl is a common 
logarithmic function. The inverse transformation function F2 is an 

10 exponential function (antilog) that functions as the inverse function of 
the common logarithmic function. The enhancement function F3 is any 
of the enhancement functions Rl to R3 explained using Fig. 49. 

These profile data allow visual processing using an input signal 
IS and an unsharp signal US that have been transformed to logarithmic 

15 space by the transformation function Fl. Human visual performance is 
logarithmic, and visual processing that is suited for our visual 
performance is best achieved by performing processing after 
transforming to logarithmic space. Due to this, the visual processing 
device 1 performs contrast enhancement in logarithmic space. 

20 It should be noted that depending on the combination of the value 

A of the input signal IS and the value B of the unsharp signal US, the 
value C obtained from Eq. M14 may be a negative value. In this case, 
the element of the profile data corresponding to the value A of the input 
signal IS and the value B of the unsharp signal US may be set to the 

25 value 0. Further, depending on the combination of the value A of the 
input signal IS and the value B of the unsharp signal US, the value C 
that is obtained from Eq. M14 may be saturated. That is, it may exceed 
the maximum value of 255 that can be expressed with 8 bits. In this 
case, the element of the profile data corresponding to the value A of the 



27 



input signal IS and the value B of the unsharp signal US may be set to 
the value 255. In Fig. 5, each element of the profile data found in this 
manner is shown by a contour line. 

Non-linear profile data are described in more specific detail in the 
5 section <Profile Data> below. 

(5) 

It is also possible for the profile data of the two-dimensional LUT 
4 to include a plurality of gradation transformation curves (gamma 
curves) for correcting the gradation of the input signal IS. 

10 Each gradation transformation curve is a monotonically 

increasing function such as a gamma function that has different gamma 
coefficients, and is associated with the value of the unsharp signal US. 
This association is performed such that gamma functions having large 
gamma coefficients are selected for the values of small unsharp signals 

15 US, for example. Thus, the unsharp signal US can function as a 
selection signal for selecting at least one gradation transformation curve 
from the gradation transformation curve group included in the profile 
data. 

According to the above configuration, the gradation of the value A 
20 of the input signal IS is transformed using a gradation transformation 
curve that has been selected by the value B of the unsharp signal US. 

It should be noted that it is also possible to interpolate the output 
of the two-dimensional LUT 4 as described in (2) above. 

(6) 

25 It was explained above that the profile data registration device 8 

is provided in or connected to the visual processing device 1, stores data 
of a plurality of profiles created in advance by a PC or the like, and can 
be used to change the content registered to the two-dimensional LUT 4. 

Here, the profile data stored in the profile data registration 
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device 8 are created by a PC that is disposed outside of the visual 
processing device 1. The profile data registration device 8 obtains the 
profile data from the PC over a network or via a recording medium. 

The profile data registration device 8 registers data of a plurality 
5 of profiles to be stored to the two-dimensional LUT 4 in accordance with 
a predetermined condition. This is described in greater detail using 
Figs. 6 to 8. It should be noted that portions having substantially the 
same function as in the visual processing device 1, which was described 
above using Fig. 1, are assigned identical reference numerals and further 
10 description thereof is omitted. 

«1» 

Fig. 6 is a block diagram of a visual processing device 520 that 
determines the image of an input signal IS, and based on the results of 
this determination, switches the profile data registered to the 
15 two-dimensional LUT 4. 

The visual processing device 520, in addition to the same 
structural components as the visual processing device 1 shown in Fig. 1, 
also has a profile data registration portion 521 that has the same 
function as the profile data registration device 8. The visual processing 
20 device 520 is further provided with an image determination portion 522. 

The image determination portion 522 receives an input signal IS 
and outputs determination results SA of the input signal IS. The profile 
data registration portion 521 receives the determination results SA and 
outputs profile data PD selected based on those determination results 
25 SA. 

The image determination portion 522 determines the image of the 
input signal IS. In this determination of the image, the brightness of 
the input signal IS is determined by obtaining a pixel value such as the 
luminance or the brightness of the input signal IS. 
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The profile data registration portion 521 obtains the 
determination results SA, and based on these determination results SA, 
it switches the profile data PD and outputs the result. More specifically, 
if the input signal IS is determined to be bright, then profile data that 
5 compress the dynamic range, for example, are selected. By doing this, 
the contrast can be maintained even for images that are bright overall. 
Also, a profile with an output signal OS having a suitable dynamic range 
for the characteristics of the device that will display the output signal 
OS is selected. 

10 Thus, the visual processing device 520 is capable of suitably 

visually processing an input signal IS. 

It should be noted that it is also possible for the image 
determination portion 522 to determine an image characteristic, such as 
the spatial frequency, in addition to a pixel value such as the luminance 

15 or the brightness of the input signal IS. 

In this case, it is possible to achieve more suitable visual 
processing, such as the selection of a profile that leads to a higher degree 
of sharpness enhancement with respect to an input signal IS whose 
spatial frequency is low. 

20 «2» 

Fig. 7 is a block diagram of a visual processing device 525 that 
switches the profile data registered to the two-dimensional LUT 4 based 
on the results of an input from an input device for inputting conditions 
relating to the brightness. 

25 The visual processing device 525, in addition to the same 

structural components as the visual processing device 1 shown in Fig. 1, 
also is provided with a profile data registration portion 526 that has the 
same function as the profile data registration device 8. The visual 
processing device 525 is provided with an input device 527 to which it is 
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connected through a wired or a wireless connection. More specifically, 
the input device 527 can be achieved for example by an input button 
provided on, or remote control for, a device that handles images, such as 
a computer, a television, a digital camera, a portable telephone, a PDA, a 
5 printer, or a scanner, that outputs an output signal OS. 

The input device 527 is an input device for inputting conditions 
related to the brightness, and for example is provided with switches such 
as "bright" and "dark." The input device 527 outputs the input results 
SB through control by a user. 

10 The profile data registration portion 526 obtains the input results 

SB, and based on the input results SB switches the profile data PD and 
outputs the result. More specifically, for example if the user has input 
"bright," then a profile for compressing the dynamic range of the input 
signal IS, for example, is selected and this is output as the profile data 

15 PD. Thus, the contrast can be maintained even if the device for 
displaying the output signal OS has been placed in a "bright" 
environment. 

Thus, with the visual processing device 525, it is possible to 
attain suitable visual processing that is in accordance with the input 
20 from the input device 527. 

It should be noted that the conditions related to the brightness 
can be not only the conditions related to the brightness of the ambient 
light around the medium that will output the output signal, such as a 
computer, digital camera, portable telephone, or PDA, but also can be the 
25 conditions related to the brightness of the medium itself to which the 
output signal will be output, such as the printer paper. The conditions 
can also be related to the brightness of the medium by which the input 
signal is received, such as the scanner paper. 

It is also possible for these conditions to be automatically input 
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by a photo sensor, for example, instead of only being input through a 
switch or the like. 

It should be noted that it is also possible for the input device 527 
to be a device that directly causes the profile data registration portion 
5 526 to change the profile rather than a device that simply inputs 
conditions related to brightness. In this case, the input device 527 can 
display a list of the profile data in addition to the conditions to the 
related to the brightness, and have the user make a selection from 
among these. 

10 By doing this, the user can perform visual processing that suits 

his personal tastes. 

It should be noted that the input device 527 can also be a device 
for authenticating the user. In this case, the input device 527 can be a 
device such as a camera that authenticates the user or a device through 

15 which the user inputs his or her user name. 

For example, if the user indicates through the input device 527 
that he or she is a child, then profile data for inhibiting extreme 
luminance changes, for example, are selected. 

Visual processing that is suited for the user thus can be achieved. 

20 «3» 

Fig. 8 is a block diagram of a visual processing device 530 that 
switches the profile data registered to the two-dimensional LUT 4 based 
on the detection results from a luminance detection portion for detecting 
two types of brightness. 

25 The visual processing device 530, in addition to the same 

structural components as the visual processing device 1 shown in Fig. 1, 
also has a profile data registration portion 531 that has the same 
function as the profile data registration device 8. The visual processing 
device 530 is further provided with a brightness detection portion 532. 
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The brightness detection portion 532 comprises the image 
determination portion 522 and the input device 527. The image 
determination portion 522 and the input device 527 are the same as 
those described using Figs. 6 and 7. Thus, the brightness detection 
5 portion 532 receives the input signal IS as input, and outputs the 
determination results SA from the image determination portion 522 and 
the input results SB from the input device 527 as detection results. 

The profile data registration portion 531 receives the 
determination results SA and the input results SB as input, and based 
10 on the determination results SA and the input results SB, switches the 
profile data PD and outputs the result. More specifically, for example if 
the ambient light is "bright" and it also has been determined that the 
input signal IS is bright, then a profile that compresses the dynamic 
range of the input signal IS, for example, is selected and this is output as 
15 the profile data PD. Thus, the contrast can be maintained when the 
output signal OS is displayed. 

Thus, with the visual processing device 530, it is possible to 
achieve suitable visual processing. 

«4» 

20 In the visual processing devices of Figs. 6 to 8, it is not necessary 

to provide the profile data registration portion in a single unit with the 
visual processing device. More specifically, the profile data registration 
portions can be connected to the visual processing devices over a network 
as a server provided with a plurality of profile data sets or as a plurality 

25 of servers each provided with profile data. Here, the network is a wired 
or wireless connection means that is capable of communication, such as a 
dedicated telephone line, a public telephone line, the Internet, or a LAN. 
In this case, the determination results SA and the input results SB both 
are transferred from the visual processing device side to the profile data 
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registration portion side over the same network. 
(7) 

In the above embodiment, it was explained that the profile data 
registration device 8 is provided with a plurality of profile data sets, and 
5 different visual processing was achieved by switching which of these is 
registered to the two-dimensional LUT 4. 

Here, it is also possible for the visual processing device 1 to be 
provided with a plurality of two-dimensional LUT, each being provided 
with profile data for achieving a different visual processing. In this 
10 case, the visual processing device 1 achieves different visual processing 
by switching which of the two-dimensional LUTs receives the input or by 
switching which of the two-dimensional LUTs to receive the output from. 

In this case, the memory capacity that must be secured for the 
two-dimensional LUT increases, but the time required for switching 
15 between the different visual processing types can be shortened. 

It is also possible for the profile data registration device 8 to be 
provided with a device that creates new profile data based on data of a 
plurality of profiles and registers the profile data that are created to the 
two-dimensional LUT 4. 
20 This implementation is described using Figs. 9 and 10. 

Fig. 9 is a block diagram that primarily illustrates a profile data 
registration device 701, as a modified example of the profile data 
registration device 8. The profile data registration device 701 is a 
device for switching the profile data that are registered to the 
25 two-dimensional LUT 4 of the visual processing device 1. 

The profile data registration device 701 comprises a profile data 
registration portion 702 to which a plurality of profile data sets are 
registered, a profile creation execution portion 703 that creates new 
profile data based on the data of a plurality of profiles, a parameter 
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input portion 706 for inputting parameters with which to create new 
profile data, and a control portion 705 that controls these portions. 

Data of a plurality of profiles are registered to the profile data 
registration portion 702 as in the profile data registration device 8 or the 
5 profile data registration portions shown in Fig. 6 to Fig. 8, and the 
profile data that have been selected through a control signal CIO from 
the control portion 705 are read out. Here, data of two selected profiles 
are read out from the profile data registration portion 702, namely first 
selected profile data dlO and second selected profile data dll. 

10 The profile data that are read from the profile data registration 

portion 702 are determined based on input from the parameter input 
portion 706. The parameter input portion 706, either manually or 
automatically from a sensor or the like, receives the desired visual 
processing effect, the degree of that processing, or information relating 

15 to the viewing environment of the image to be processed, as a parameter. 
From the parameter input by the parameter input portion 706, the 
control portion 705 specifies the profile data to be read out with a control 
signal clO and also specifies the value of the degree of synthesis of the 
respective profile data using a control signal cl2. 

20 The profile creation execution portion 703 is provided with a 

profile creation portion 704 that produces a created profile data d6, 
which are new profile data, from the first selected profile data dlO and 
the second selected profile data dll. 

The profile creation portion 704 obtains the first selected profile 

25 data dlO and the second selected profile data dll from the profile data 
registration portion 702. It also obtains the control signal cl2, which 
specifies the degree of synthesis of the respective selected profile data, 
from the control portion 705. 

The profile creation portion 704 then applies the value [k] of the 
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degree of synthesis specified by the control signal cl2 to a value [m] of 
the first selected profile data dlO and a value [n] of the second selected 
profile data dll to create created profile data d6 having a value [1]. 
Here, the value [1] is calculated by [1] = (l-k) * [m]+k * [n]. It should be 
5 noted that if the value [k] is 0<k<l, then the first selected profile data 
dlO and the second selected profile data dll are interpolated, and if the 
value [k] is k<0 or k>l, then the first selected profile data dlO and the 
second selected profile data dll are extrapolated. 

The two-dimensional LUT 4 obtains the created profile data d6 

10 created by the profile creation portion 704 and stores the obtained value 
in an address that is specified by a count signal ell of the control portion 
705. Here, the created profile data d6 are associated with the same 
image signal value as the selected profile data that were used to create 
the created profile data d6. 

15 Thus, profile data that achieve different visual processing can be 

used to create new profile data that achieve yet different visual 
processing. 

The visual processing profile creation method that is executed by 
the visual processing device provided with the profile data registration 

20 device 701 is described using Fig. 10. 

The address of the profile data registration portion 702 is 
specified at a fixed count period by a count signal clO from the control 
portion 705, and the image signal value that is stored in the specified 
address is read out (step S70l). More specifically, the control portion 

25 705 outputs a count signal clO in correspondence with a parameter that 
has been input by the parameter input portion 706. The count signal 
clO specifies the addresses of two profile data, each of which achieves 
different visual processing, in the profile data registration portion 702. 
From this signal, the first selected profile data dlO and the second 
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selected profile data dll are read from the profile data registration 
portion 702. 

The profile creation portion 704 obtains the control signal cl2, 
which specifies the degree of synthesis, from the control portion 705 
5 (step S702). 

The profile creation portion 704 then applies the value [k] of the 
degree of synthesis specified by the control signal cl2 to the value [m] of 
the first selected profile data dlO and the value [n] of the second selected 
profile data dll to create a created profile data d6 having the value [1] 
10 (step S703). Here, the value [1] is calculated by [1] = (l-k) * [m]+k * [ n ]. 

The created profile data d6 are written to the two-dimensional 
LUT 4 (step S704). The address to which the data are written is 
specified by a count signal ell from the control portion 705 that is given 
to the two-dimensional LUT 4. 
15 The control portion 705 determines whether or not processing has 

finished for all of the selected profile data (step S705), and repeats the 
processes of step S701 to step S705 until this processing is finished. 

The new profile data stored in the two-dimensional LUT 4 in this 
manner are then used to execute visual processing. 
20 «Effect of (7)» 

With the visual processing device provided with the profile data 
registration device 701, profile data that achieve different visual 
processing can be used to create new profile data that achieve yet 
different visual processing, and this visual processing can be performed. 
25 That is, it is possible to achieve visual processing to any degree simply 
by providing the profile data registration portion 702 with a small 
number of profile data, and thus the memory capacity of the profile data 
registration portion 702 can be reduced. 

It should be noted that the profile data registration device 701 
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can be provided not only in the visual processing device 1 shown in Fig. 1, 
but also in any of the visual processing devices of Figs. 6 to 8. In this 
case, the profile data registration portion 702 and the profile creation 
execution portion 703 are used in place of the profile data registration 
5 portions 521, 526, and 531 shown in Figs. 6 to 8, respectively, and the 
parameter input portion 706 and the control portion 705 are used in 
place of the image determination portion 522 of Fig. 6, the input device 
527 of Fig. 7, and the brightness detection portion 532 of Fig. 8. 
(8) 

10 The visual processing device can also be a device that transforms 

the brightness of the input signal IS. A visual processing device 901 
that transforms the brightness is described using Fig. 11. 
«Configuration» 

The visual processing device 901 is a device that transforms the 
15 brightness of an input signal IS', and includes a processing portion 902 
that performs a predetermined processing with respect to the input 
signal IS' and outputs a processed signal US', and a transformation 
portion 903 that uses the input signal IS' and the processed signal US' to 
perform transformation of the input signal IS'. 
20 The processing portion 902 operates in the same manner as the 

spatial processing portion 2 (see Fig. l), and performs spatial processing 
of the input signal IS'. It should be noted that it can also perform 
spatial processing such as that discussed in the above <Modified 
Example> (3). 

25 The transformation portion 903 is provided with a 

two-dimensional LUT like the visual processing portion 3 and outputs an 
output signal OS' (value [y]) based on the input signal IS' (value [x]) and 
the processed signal US' (value [z]). 

Here, the values of the elements of the two-dimensional LUT of 
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the transformation portion 903 are determined by applying the value [x] 
of the input signal IS' to gain or offset that has been determined in 
correspondence with the value of the function fk(z), which is related to 
the extent of change in brightness. The function fk(z) related to the 
5 extent of change in brightness is hereinafter referred to as the "extent 
change function." 

The value of each element of the two-dimensional LUT (=the 
value [y] of the output signal OS') is determined according to a function 
relating the value [x] of the input signal IS' and the value [z] of the 
10 processed signal US'. Hereinafter, this function is referred to as a 
"transformation function." and the transformation functions (a) to (d) are 
examples thereof. Figs. 12(a) to (d) show the relationship between the 
input signal IS' and the output signal OS' when the extent change 
function fk(z) has been changed. 
15 «Regarding the Transformation Function (a)» 

The transformation function (a) is expressed as [y] = fl(z) *[x]. 

Here, the extent change function fl(z) acts as gain of the input 
signal IS'. Thus, the gain of the input signal IS' changes depending on 
the value of the extent change function fl(z), and this changes the value 
20 [y] of the output signal OS'. 

Fig. 12(a) shows the change in the relationship between the input 
signal IS' and the output signal OS' for various values of the extent 
change function fl(z). 

The value [y] of the output signal increases as the extent change 
25 function fl(z) becomes larger (fl(z)>l). That is, the image after 
transformation becomes brighter. On the other hand, the value [y] of 
the output signal decreases as the extent change function fl(z) becomes 
smaller (fl(z)<l). That is, the image after transformation becomes 
darker. 
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Here, the extent change function fl(z) is a function in which the 
smallest value in the defined domain of the value [z] does not become 
less than [0]. 

If due to the computation of the transformation function (a) the 
5 value [y] of the output signal is outside the valid range of values, then it 
may be clipped to the valid range of values. For example, if the value 
[y] of the output signal exceeds the value [l], then it is possible to clip it 
to the value [l], and if the value [y] of the output signal is below the 
value [0], then it can be clipped to the value [0]. This also applies to the 
10 other transformation functions (b) through (d) discussed below. 
«Regarding the Transformation Function (b)» 
The transformation function (b) is expressed as [y] = [x] + f2(z). 
Here, the extent change function f2(z) acts as offset of the input 
signal IS'. Thus, the offset of the input signal IS' changes depending on 
15 the value of the extent change function f2(z), and this changes the value 
[y] of the output signal OS'. 

Fig. 12(b) shows the change in the relationship between the input 
signal IS' and the output signal OS' when the value of the extent change 
function f2(z) is changed. 
20 The value [y] of the output signal increases as the extent change 

function f2(z) becomes larger (f2(z)>0). That is, the image after 
transformation becomes brighter. On the other hand, the value [y] of 
the output signal decreases as the extent change function f2(z) becomes 
smaller (f2(z)<0). That is, the image after transformation becomes 
25 darker. 

«Regarding the Transformation Function (c)» 

The transformation function (c) is expressed as [y] = fl(z) * [x] + 

f2(z). 

Here, the extent change function fl(z) acts as gain of the input 
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signal IS'. The extent change function f2(z) also acts as offset of the 
input signal IS'. Thus, the gain of the input signal IS' changes 
depending on the value of the extent change function fl(z), and the offset 
of the input signal IS' changes depending on the value of the extent 
5 change function f2(z), and this changes the value [y] of the output signal 
OS'. 

Fig. 12(c) shows the change in the relationship between the input 
signal IS' and the output signal OS' when the values of the extent change 
function fl(z) and the extent change function f2(z) are changed. 
10 The value [y] of the output signal increases as the extent change 

function fl(z) and the extent change function f2(z) become larger. That 
is, the image after transformation becomes brighter. On the other hand, 
the value [y] of the output signal decreases as the extent change function 
fl(z) and the extent change function f2(z) become smaller. That is, the 
15 image after transformation becomes darker. 

«Regarding the Transformation Function (d)» 
The transformation function (d) is expressed as [y] = [x] 
Ml-f2(z)). 

Here, the extent change function f2(z) determines the "power" of 
20 the "power function." Thus, the input signal IS' changes depending on 
the value of the extent change function f2(z), and this changes the value 
[y] of the output signal OS'. 

Fig. 12(d) shows the change in the relationship between the input 
signal IS' and the output signal OS' when the value of the extent change 
25 function f2(z) is changed. 

The value [y] of the output signal increases as the extent change 
function f2(z) becomes larger (f2(z)>0). That is, the image after 
transformation becomes brighter. On the other hand, the value [y] of 
the output signal decreases as the extent change function f2(z) becomes 
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smaller (f2(z)<0). That is, the image after transformation becomes 
darker. If the extent change function f2(z) is the value [0], then 
transformation with respect to the input signal IS' is not performed. 

It should be noted that the value [x] is a value obtained by 
5 normalizing the value of the input signal IS' to the range of [0] to [l]. 

«Effect» 

(1) 

With the visual processing device 901, visual processing of the 
input signal IS' is performed in accordance with a two-dimensional LUT 
10 that has elements that have been determined using one of the 
transformation functions (a) to (d) shown above. Each element of the 
two-dimensional LUT stores a value [y] with respect to a value [x] and a 
value [z]. Thus, it is possible to achieve viewing angle processing in 
which the brightness of the input signal IS' is transformed based on the 
15 input signal IS' and the processed signal US'. 
(2) 

Here, if the extent change function fl(z) and the extent change 
function f2(z) are both functions that decrease monotonically, then it is 
also possible to achieve the effect of backlight correction or to prevent 

20 blooming, for example. This is described in greater detail below. 

Figs. 13(a) and (b) show examples in which the extent change 
function fl(z) and the extent change function f2(z) both decrease 
monotonically. Each shows three graphs (al to a3, bl to b3), and each of 
these graphs is an example of a function that decreases monotonically. 

25 The extent change function fl(z) is a function that has a value 

range that spans the value [l], and is a function in which the minimum 
value with respect to the defined range of the value [z] is not less than 
the value [0]. The extent change function f2(z) is a function that has a 
value range that spans the value [0]. 
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For example, in portions in the image in which the area of 
darkness is large, the value [z] of the processed signal US' is small. The 
value of the extent change function for a small value [z] is large. That 
is, when a two-dimensional LUT that has been created based on a 
5 transformation function (a) through (d) is used, then portions in the 
image in which the area of darkness is large are made brighter. Thus, 
for example in an image that is captured with backlight, the dark 
portions are fixed in sections in which the area of darkness is large, and 
this improves the visual effect. 

10 Also, for example in portions in the image in which the area of 

brightness is large, the value [z] of the processed signal US' is large. 
The value of the extent change function for a large value [z] is small. 
That is, when a two-dimensional LUT that has been created based on a 
transformation function (a) through (d) is used, portions in the image in 

15 which the area of brightness is large are made darker. Thus, for 
example in an image having bright portions such as the sky, blooming is 
fixed in the portions in which the area of brightness is large, and this 
improves the visual effect. 

«<Modified Examples»> 

20 (1) 

The transformation functions described above are only examples, 
and any functions are possible as long as transformation having the 
same characteristics is achieved. 

(2) 

25 It is not absolutely necessary that the value of each element of 

the two-dimensional LUT is determined by a transformation function 
discussed above. 

For example, if the value of the above transformation functions is 
outside the range of values that can be adopted as the output signal OS', 
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then the two-dimensional LUT can store values that have been clipped to 
a range of values that can be adopted as the output signal OS'. 
(3) 

It is also possible to perform the same processing as discussed 
5 above without using a two-dimensional LUT. For example, it is also 
possible for the transformation portion 903 to execute the transformation 
functions (a) to (d) with respect to the input signal IS' and the processed 
signal US' and output the result as the output signal OS'. 

(9) 

10 It is also possible for the visual processing device to be provided 

with a plurality of spatial processing portions and for it to perform visual 
processing using a plurality of unsharp signals each spatially processed 
to a different degree. 

«Configuration» 

15 Fig. 14 shows the configuration of a visual processing device 905. 

The visual processing device 905 is a device that performs visual 
processing of an input signal IS", and comprises a first processing 
portion 906a that performs a first predetermined processing with respect 
to the input signal IS" and outputs a first processed signal Ul, a second 

20 processing portion 906b that performs a second predetermined 
processing with respect to the input signal IS" and outputs a second 
processed signal U2, and a transformation portion 908 that uses the 
input signal IS", the first processed signal Ul, and the second processed 
signal U2 to perform transformation of the input signal IS". 

25 The first processing portion 906a and the second processing 

portion 906b both operate in the same manner as the spatial processing 
portion 2 (see Fig. l), and perform spatial processing of the input signal 
IS". It should be noted that it is also possible for them to perform 
spatial processing such as that described in the <Modified Example> (3) 
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above. 

Here, the first processing portion 906a and the second processing 
portion 906b differ in the size of the region of the surrounding pixels that 
are used for the spatial processing. 
5 More specifically, the first processing portion 906a uses the 

surrounding pixels that are included in a region of 30 pixels vertically by 
30 pixels horizontally with the target pixel in the center (small unsharp 
signal), whereas the second processing portion 906b uses the 
surrounding pixels that are included in the region of 90 pixels vertically 

10 by 90 pixels horizontally with the target pixel in the center (large 
unsharp signal). It should be noted that the region of surrounding 
pixels that is described here is only one example, and there is no 
limitation to this. It is preferable that the unsharp signal is created 
from a fairly wide region in order to obtain a sufficient visual processing 

15 effect. 

The transformation portion 908 is provided with a LUT, and 
outputs an output signal OS" (value [y]) based on the input signal IS" 
(value [x]), the first processed signal Ul (value [zl]), and the second 
processed signal U2 (value [z2]). 

20 Here, the LUT that the transformation portion 903 is provided 

with is a three-dimensional LUT that stores values [y] of the output 
signal OS" with respect to the values [x] of the input signal IS", the 
values [zl] of the first processed signal Ul, and the values [z2] of the 
second processed signal U2. The value of each element of this 

25 three-dimensional LUT (= the values [y] of the output signal OS") is 
determined based on a function of the value [x] of the input signal IS', 
the value [zl] of the first processed signal Ul, and the value [z2] of the 
second processed signal U2. 

This three-dimensional LUT is capable of achieving the 
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processing discussed in the above-described and later-described 
embodiments, but here the three-dimensional LUT is described for the 
«Case of Transforming the Brightness of the Input Signal IS"» and 
the «Case of Enhancement Transformation of the Input Signal IS"». 
5 «Case of Transforming the Brightness of the Input Signal IS"» 

The transformation portion 908 performs a transformation to 
brighten the input signal IS" if the value [zl] of the first processed signal 
Ul is small. However, the extent to which it is brightened is kept low if 
the value [z2] of the second processed signal US also is small. 
10 As examples of this transformation, the value of each element of 

the three-dimensional LUT of the transformation portion 903 is 
determined based on the following transformation function (e) or (f). 

(Regarding the Transformation Function (e)) 

The transformation function (e) is expressed by [y] = 
15 [fll(zl)/fl2(z2)] * [x]. 

Here, the extent change functions fll(zl) and fl2(z2) are the 
same function as the extent change function fl(z) described in <Modified 
Example> (8) above. The extent change function fll(zl) and the extent 
change function fl2(z2) are different functions. 
20 Thus, [fll(zl)/fl2(z2)] acts as the gain of the input signal IS", and 

the gain of the input signal IS" changes in accordance with the value of 
the first processed signal Ul and the value of the second processed signal 
U2, and this changes the value [y] of the output signal OS". 
(Regarding the Transformation Function (f)) 
25 The transformation function (f) is expressed by [y] = [x] + f2l(zl) 

-f22(z2). 

Here, the extent change functions f2l(zl) and f22(z2) are the 
same function as the extent change function f2(z) described in <Modified 
Example> (8) above. The extent change function f2l(zl) and the extent 
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change function f22(z2) are different functions. 

Thus, [f2l(zl) - f22(z2)] acts as the offset of the input signal IS", 
and the offset of the input signal IS" changes in accordance with the 
value of the first processed signal Ul and the value of the second 
5 processed signal U2, and this changes the value [y] of the output signal 
OS". 

(Effect) 

Through transformation using the transformation functions (e) 
and (f), it is possible to achieve the effect of brightening dark areas in 
10 regions having a small area of backlight while keeping the large dark 
area region of nighttime images from being brightened too much. 

(Modified Example) 

It should be noted that the processing by the transformation 
portion 908 is not limited to processing using a three-dimensional LUT, 
15 and it can also be processing in which the same computations as those of 
the transformation functions (e) or (f), for example, are performed. 

Also, it is not absolutely necessary that the elements of the 
three-dimensional LUT are determined based on the transformation 
functions (e) or (f). 

20 «Case of Enhancement Transformation of the Input Signal 

IS"» 

If the transformation by the transformation portion 908 is 
transformation that enhances the input signal IS", then it is possible to 
independently enhance a plurality of frequency components. 
25 In the case of a transformation further enhancing the first 

processed signal Ul, it is possible to enhance dark portions in which the 
frequency is relatively high, whereas in the case of a transformation 
further enhancing the second processed signal U2, it is possible to 
enhance dark portions in which the frequency is low. 
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<Profile Data> 

The visual processing device 1 can also be provided with profile 
data for achieving various types of visual processing other than that 
described above. First through seventh profile data for achieving 
5 various types of visual processing are described below, for each showing 
the equation that characterizes the profile data and the structure of a 
visual processing device that achieves visual processing that is 
equivalent to the visual processing device 1 provided with those profile 
data. 

10 Each set of profile data is determined based on a mathematical 

equation that includes a computation for enhancing the values calculated 
from the input signal IS and the unsharp signal US. Here, this 
computation for enhancing is a computation that is achieved by a 
non-linear enhancement function, for example. 

15 Thus, with each set of profile data, it is possible to achieve 

enhancement that is suited for the visual characteristics of the input 
signal IS or enhancement that is suited for the non-linear characteristics 
of the device that will output the output signal OS. 
(1) 

20 «First Profile Data» 

The first profile data are determined in accordance with a 
computation that includes a function for enhancing the difference 
between the respective transformed values that are obtained by 
performing a predetermined transformation with respect to the input 

25 signal IS and the unsharp signal US. By doing this, it becomes possible 
to enhance the respective differences after transforming the input signal 
IS and the unsharp signal US in separate spaces. Thus, it is possible to 
achieve enhancement that is suited for the visual characteristics, for 
example. 
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Hereinafter, this will be described in more detail. 
The values C of the elements of the first profile data (the value of 
the output signal OS) are expressed by C = F2(Fl(A)+F3(Fl(A)-Fl(B))) 
(hereinafter, this is referred to as Equation Ml) using the value A of the 
5 input signal IS, the value B of the unsharp signal US, the 
transformation function Fl, the inverse transformation function F2 of 
the transformation function, and an enhancement function F3. 

Here, the transformation function Fl is a common logarithmic 
function. The inverse transformation function F2 is an exponential 
10 function (antilog) that functions as the inverse function of the common 
logarithmic function. The enhancement function F3 is any of the 
enhancement functions Rl to R3 explained using Fig. 49. 
«Equivalent Visual Processing Device 11» 
Fig. 15 shows a visual processing device 11 that is equivalent to 
15 the visual processing device 1 in which the first profile data have been 
registered to the two-dimensional LUT 4. 

The visual processing device 11 is a device that outputs an output 
signal OS based on a computation that enhances the differences in the 
transformed values of the input signal IS and the unsharp signal US 
20 after the two have been subjected to a predetermined transformation. 
Thus, the input signal IS and the unsharp signal US are transformed to 
separate spaces and their respective differences are enhanced, and this 
allows for example enhancement that is suited for the visual 
characteristics to be achieved. 
25 The visual processing device 11 shown in Fig. 15 is provided with 

a spatial processing portion 12 that performs spatial processing on the 
luminance of each pixel of an original image that it has obtained as an 
input signal IS and outputs an unsharp signal US, and a visual 
processing portion 13 that uses the input signal IS and the unsharp 
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signal US to perform visual processing of the original image and outputs 
an output signal OS. 

The spatial processing portion 12 performs the same operation as 
the spatial processing portion 2 of the visual processing device 1, and 
5 thus its further description is omitted. 

The visual processing portion 13 is provided with a signal space 
transformation portion 14 that performs transformation of the signal 
space of the input signal IS and the unsharp signal US and outputs a 
transformed input signal TIS and a transformed unsharp signal TUS, a 

10 subtracting portion 17 that receives the transformed input signal TIS as 
a first input and the transformed unsharp signal TUS as a second input 
and outputs a difference signal DS of the difference between the two, an 
enhancing portion 18 that receives the difference signal DS as input and 
outputs an enhanced signal TS that has been enhanced, a summing 

15 portion 19 that receives the transformed input signal TIS as a first input 
and the enhanced signal TS as a second input and sums the two to 
obtain a summed signal PS that it then outputs, and an inverse 
transformation portion 20 that receives the summed signal PS as an 
input and outputs an output signal OS. 

20 The signal space transformation portion 14 further includes a 

first transformation portion 15 that receives the input signal IS as input 
and outputs the transformed input signal TIS, and a second 
transformation portion 16 that receives the unsharp signal US as input 
and outputs the transformed unsharp signal TUS. 

25 «Action of the Equivalent Visual Processing Device 11» 

The operation of the visual processing portion 13 is described in 
further detail below. 

The first transformation portion 15 uses the transformation 
function Fl to transform the input signal, which has the value A, to the 
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transformed input signal TIS, which has the value Fl(A). The second 
transformation portion 16 uses the transformation function Fl to 
transform the unsharp signal US, which has the value B, to the 
transformed unsharp signal TUS, which has the value Fl(B). The 
5 subtracting portion 17 calculates the difference between the transformed 
input signal TIS having the value Fl(A) and the transformed unsharp 
signal TUS having the value Fl(B) and outputs a difference signal DS 
having the value Fl(A) - Fl(B). The enhancing portion 18 uses the 
enhancement function F3 to output an enhanced signal TS having a 

10 value F3(F1(A) - Fl(B)) from the difference signal DS of the value Fl(A) 
- Fl(B). The summing portion 19 takes the sum of the transformed 
input signal TIS having the value Fl(A) and the enhanced signal TS 
having the value F3(Fl(A) - Fl(B)) and outputs a summed signal PS 
that has a value Fl(A) + F3(Fl(A) - Fl(B)). The inverse transformation 

15 portion 20 uses the inverse transformation function F2 to perform 
inverse transformation of the summed signal PS having the value Fl(A) 
+ F3(F1(A) - Fl(B)), and outputs an output signal OS having the value 
F2(F1(A) + F3(F1(A) - Fl(B))). 

It should be noted that the calculations using the transformation 

20 function Fl, the inverse transformation function F2, and the 
enhancement function F3 each can be performed using a 
one- dimensional LUT for that function, or can be performed without 
using a LUT. 

«Effects» 

25 The visual processing device 11 and the visual processing device 1 

provided with the first profile data attain the same visual processing 
effects. 

(i) 

Visual processing using the transformed input signal TIS and the 
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transformed unsharp signal TUS, which were transformed to logarithmic 
space by the transformation function Fl, is achieved. Human visual 
characteristics are logarithmic, and visual processing that is suited for 
our visual characteristics is achieved by performing processing after 
5 transforming to logarithmic space, 
(ii) 

With each of these visual processing devices, contrast 
enhancement in the logarithmic space is achieved. 

The conventional visual processing device 400 shown in Fig. 48 is 

10 used for performing edge enhancement using the unsharp signal US, 
which generally has a small degree of blur. However, if the visual 
processing device 400 performs contrast enhancement using an unsharp 
signal US that has a large degree of blur, then bright portions in the 
original image are not enhanced enough and dark portions in the 

15 original image are enhanced too much, resulting in visual processing 
that is not suited for our visual characteristics. That is, the visual 
processing device 400 has a tendency for insufficient enhancement when 
correcting by increasing the brightness and too much enhancement when 
correcting by increasing the darkness. 

20 On the other hand, if visual processing is performed using the 

visual processing device 1 or the visual processing device 11, it is 
possible to perform visual processing that is suited for our visual 
characteristics over a range that spans dark and light areas, and 
enhancement for increased brightness and enhancement for increased 

25 darkness can be performed with good balance, 
(iii) 

With the conventional visual processing device 400, there are 
cases where the output signal OS is negative after visual processing, and 
fails. 
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On the other hand, if the value C of a particular element of the 
profile data found with Eq. Ml is outside of the range 0<C<255, then the 
value of that element can be set to either 0 or 255 in order to prevent the 
pixel signals after correction from becoming negative and failing or 
5 becoming saturated and failing. This can be achieved regardless of the 
bit length for expressing the elements of the profile data. 
«Modified Examples» 
(i) 

The transformation function Fl is not limited to a logarithmic 
10 function. For example, it is also possible for the transformation 
function Fl to achieve transformation that removes the gamma 
correction (for example, a gamma coefficient of "0.45") that is applied to 
the input signal IS, and for the inverse transformation function F2 to 
achieve transformation that applies the gamma correction that has been 
15 applied to the input signal IS. 

The gamma correction that is applied to the input signal IS can 
thus be removed so as to perform processing under linear conditions. 
Thus, it is possible to correct optical blurring, 
(ii) 

20 With the visual processing device 11, it is also possible for the 

visual processing portion 13 to execute the computation of Eq. Ml, 
without using the two-dimensional LUT 4, according to the input signal 
IS and the unsharp signal US. In this case, it is possible to use a 
one-dimensional LUT for the calculations of the functions Fl to F3. 

25 (2) 

«Second Profile Data» 
The second profile data are determined by a computation that 
includes a function for enhancing the ratio between the input signal IS 
and the unsharp signal US. Thus, it is for example possible to achieve 
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visual processing that enhances the sharp component. 

Further, the second profile data are determined by a computation 
in which dynamic range compression is performed with respect to the 
ratio between the enhanced input signal IS and the unsharp signal US. 
5 Thus, it is for example possible to achieve visual processing in which the 
sharp component is enhanced while the dynamic range is compressed. 

This is described in greater detail below. 

The values C of the elements of the second profile data (the 
values of the output signal OS) are expressed by C = F4(A) * F5(A/B) 
10 (hereinafter, this is referred to as Equation M2) using the value A of the 
input signal IS, the value B of the unsharp signal US, a dynamic range 
compression function F4, and an enhancement function F5. 

Here, the dynamic range compression function F4 is for example 
a monotonically increasing function such as a concave power function. 
15 For example, it is expressed by F4(x) = x A y (0<y<l). The enhancement 
function F5 is a power function. It is for example expressed by F5(x) = 
x A a (0«x<l). 

«Equivalent Visual Processing Device 21» 
Fig. 16 shows a visual processing device 21 that is equivalent to 
20 the visual processing device 1 in which the second profile data have been 
registered to the two-dimensional LUT 4. 

The visual processing device 21 is a device that outputs an output 
signal OS based on a computation that enhances the ratio between the 
input IS and the unsharp signal US. Thus, it is for example possible to 
25 achieve visual processing for enhancing the sharp component. 

The visual processing device 21 also outputs an output signal OS 
based on a computation in which dynamic range compression is 
performed with respect to the ratio between the enhanced input signal IS 
and the unsharp signal US. Thus, it is for example possible to achieve 



54 



visual processing in which the sharp component is enhanced while the 
dynamic range is compressed. 

The visual processing device 21 shown in Fig. 16 is provided with 
a spatial processing portion 22 that executes spatial processing on the 
5 luminance value of each pixel of an original image that it has obtained as 
an input signal IS and outputs an unsharp signal US, and a visual 
processing portion 23 that uses the input signal IS and the unsharp 
signal US to perform visual processing of the original image and outputs 
an output signal OS. 
10 The spatial processing portion 22 performs the same operation as 

the spatial processing portion 2 of the visual processing device 1, and 
thus it will not be described further. 

The visual processing portion 23 is provided with a dividing 
portion 25 that takes the input signal IS as a first input and the unsharp 
15 signal US as a second input, and divides the input signal IS by the 
unsharp signal US to obtain a divided signal RS that it then outputs, an 
enhancing portion 26 that receives the divided signal RS as input and 
outputs an enhanced signal TS, and an output portion 27 that receives 
the input signal IS as a first input and the enhanced signal TS as a 
20 second input and outputs an output signal OS. The output portion 27 is 
provided with a DR compression portion 28 that receives the input signal 
IS as input and outputs a DR compressed signal DRS that has been 
subjected to dynamic range (DR) compression, and a multiplying portion 
29 that receives the DR compressed signal DRS as a first input and the 
25 enhanced signal TS as a second input and outputs an output signal OS. 

«Action of the Equivalent Visual Processing Device 21» 
The operation of the visual processing portion 23 is described in 
further detail below. 

The dividing portion 25 divides the input signal IS, which has the 
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value A, by the unsharp signal US, which has the value B, and outputs a 
divided signal RS having the value A/B. The enhancing portion 26 uses 
the enhancement function F5 to output an enhanced signal TS having 
the value F5(A/B) from the divided signal RS having the value A/B. The 
5 DR compression portion 28 uses the dynamic range compression function 
F4 to output a DR compressed signal DRS having the value F4(A) from 
the input signal IS, whose value is A. The multiplying portion 29 
multiplies the DR compressed signal DRS, whose value is F4(A), and the 
enhanced signal TS, whose value is F5(A/B), and outputs an output 
10 signal OS having the value F4(A) * F5(A/B). 

It should be noted that the calculations using the dynamic range 
compression function F4 and the enhancement function F5 can each be 
performed using a one-dimensional LUT for that function, or can be 
performed without using a LUT. 
15 «Effects» 

The visual processing device 21 and the visual processing device 
1 provided with the second profile data achieve the same visual 
processing effects, 
(i) 

20 Conventionally, when compressing the overall dynamic range of 

an image, the dynamic range compression function F4 shown in Fig. 17 
is used to compress the grey level from dark portions to highlights 
without causing saturation. That is, when L0 is the target black level 
to be reproduced in the image signal before compression and LI is the 

25 maximum white level, then the dynamic range LLLO prior to 
compression is compressed to the dynamic range Ql : Q0 after 
compression. However, the contrast, which is the ratio of the image 
signal level, is lowered to (Q1/Q0) * (LO/Ll) times its value due to 
dynamic range compression. Here, the dynamic range compression 
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function F4 is a concave power function, for example. 

However, with the visual processing device 21 and the visual 
processing device 1 that has been provided with the second profile data, 
the divided signal RS having the value A/B, that is, the sharp signal 
5 enhanced by the enhancement function F5, is multiplied by the DR 
compressed signal DRS. The result is that local contrast is enhanced. 
Here, the enhancement function F5 is a power function such as that 
shown in Fig. 18 (F5(x) = x A a), and leads to enhancement in the bright 
direction when the value of the divided signal RS is greater than 1 and 
10 to enhancement in the dark direction when it is less than 1. 

In general, human vision has the characteristic that an image 
will be viewed at a same contrast as long as local contrast is maintained, 
even if there is a drop in the overall contrast. Thus, with the visual 
processing device 21 and the visual processing device 1 that has been 
15 provided with the second profile data, it is possible to achieve visual 
processing with which the dynamic range is compressed but the visual 
contrast is not lowered, 
(ii) 

The effects of the present invention are described in further 
20 detailed below. 

The dynamic range compression function F4 shall be F4(x) = x A y 
(y=0.6). The enhancement function F5 shall be F5(x) = x A a (a=0.4). 
Further, if the maximum white level of the input signal is normalized to 
1, then the target black level to be reproduced shall be the value 1/300. 
25 That is, the dynamic range of the input signal IS shall be 300:1. 

If the dynamic range of this input signal IS is compressed using 
the dynamic range compression function F4, then the dynamic range 
after compression becomes F4(l):F4(l/300) = 30 ; 1. That is, the dynamic 
range is compressed by 1/10 due to the dynamic range compression 
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function F4. 

On the other hand, the value C of the output signal OS is 
expressed by the above equation M2, where C = (A A 0.6) * {(A/B) A 0.4} , 
that is, C = A/(B A 0.4). Here, in a local range the value of B is regarded 
5 as constant and thus C is proportional to A. That is, ratio of the amount 
of change of the value C and the amount of change of the value A is 1, 
and the local contrast does not change between the input signal IS and 
the output signal OS. 

As mentioned above, human vision has the characteristic that as 

10 long as local contrast is maintained, an object will be viewed at the same 
contrast even if there is a drop in the overall contrast. Thus, with the 
visual processing device 21 and the visual processing device 1 that has 
been provided with the second profile data, it is possible to achieve 
visual processing with which the dynamic range is compressed but 

15 visually the contrast is not lowered. 

It should be noted that if the power multiplier a of the 
enhancement function Fo shown in Fig. 18 is larger than 0.4, then it is 
possible to raise the visible contrast of the output signal OS over the 
input signal IS while performing compression of the dynamic range. 

20 (iii) 

Because the above effects can be achieved with the present 
invention, the invention is particularly useful in the following conditions. 
That is, it is possible to reproduce a high contrast image on a display 
having a narrow physical dynamic range without squashing the dark and 

25 light areas. It is also possible for example to display high contrast video 
on a television projector in a bright environment and to obtain a high 
contrast print using thin inks (on a printer that can print light colors 
only). 

«Modified Examples» 
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(i) 

With the visual processing device 21, it is also possible for the 
visual processing portion 23 to perform the calculation of Eq. M2 based 
on the input signal IS and the unsharp signal US without using the 
5 two-dimensional LUT 4. In this case, it is possible to use a 
one-dimensional LUT for the calculation of each of the functions F4 and 
F5. 

(ii) 

It should be noted that if the value C of a particular element of 
10 the profile data obtained by the Eq. M2 is C>255, then the value C of 
that element can be regarded as 255. 

(3) 

«Third Profile Data» 
15 The third profile data are determined based on a computation 

that includes a function for enhancing the ratio between the input signal 
IS and the unsharp signal US. Thus, it is for example possible to 
achieve visual processing that enhances the sharp component. 
This is described in further detail below. 
20 In Eq. M2 of the second profile data, it is also possible for the 

dynamic range compression function F4 to be a direct proportion 
function of a proportional coefficient 1. In this case, the value C of each 
element of the third profile data (the value of the output signal OS) is 
expressed by C = A * F5(A/B) (hereinafter, this is referred to as Equation 
25 M3) using the value A of the input signal IS, the value B of the unsharp 
signal US, and the enhancement function F5. 

«Equivalent Visual Processing Device 31» 
Fig. 19 shows a visual processing device 31 that is equivalent to 
the visual processing device 1 in which the third profile data have been 
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registered to the two-dimensional LUT 4. 

The visual processing device 31 is a device that outputs an output 
signal OS based on a computation that enhances the ratio between the 
input IS and the unsharp signal US. Thus, it is for example possible to 
5 achieve visual processing for enhancing the sharp component. 

The visual processing device 31 shown in Fig. 19 differs from the 
visual processing device 21 shown in Fig. 16 in that it is not provided 
with the DR compression portion 28. The visual processing device 31 
shown in Fig. 19 is described below, assigning the same reference 
10 numerals to and omitting description of portions that perform the same 
operations as those of the visual processing portion 21 shown in Fig. 16. 

The visual processing device 31 is provided with a spatial 
processing portion 22 that performs spatial processing of the luminance 
value of each pixel of an original image that it has obtained as an input 
15 signal IS and outputs an unsharp signal US, and a visual processing 
portion 32 that uses the input signal IS and the unsharp signal US to 
perform visual processing of the original image and outputs an output 
signal OS. 

The spatial processing portion 22 performs the same operation as 
20 the spatial processing portion 2 of the visual processing device 1, and 
thus it will not be described further. 

The visual processing portion 32 is provided with a dividing 
portion 25 that takes an input signal IS as a first input and an unsharp 
signal US as a second input, and divides the input signal IS by the 
25 unsharp signal US to obtain a divided signal RS that it then outputs, an 
enhancing portion 26 that receives the divided signal RS as input and 
outputs an enhanced signal TS, and a multiplying portion 33 that 
receives the input signal IS as a first input and the enhanced signal TS 
as a second input and outputs an output signal OS. 
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«Action of the Equivalent Visual Processing Device 31» 
The operation of the visual processing portion 32 is described in 
further detail below. 

The dividing portion 25 and the enhancing portion 26 perform the 
5 same operations as described with regard to the visual processing device 
21 shown in Fig. 16. 

The multiplying portion 33 multiplies the input signal IS, whose 
value is A, and the enhanced signal TS, whose value is value F5(A/B), 
and outputs an output signal OS having the value A * F5(A/B). Here, 
10 the enhancement function F5 is the same as that shown in Fig. 18. 

It should be noted that the calculation using the enhancement 
function F5, as described with regard to the visual processing device 21 
shown in Fig. 16, can be performed using a one-dimensional LUT for that 
function, or can be performed without using a LUT. 
15 <Effects> 

The visual processing device 31 and the visual processing device 
1 that has been provided with the third profile data achieve the same 
visual processing effect, 
(i) 

20 With the enhancing portion 26, enhancement of the sharp signal 

(divided signal RS), which is expressed as the ratio of the input signal IS 
to the unsharp signal US, is performed, and the enhanced sharp signal 
and the input signal IS are multiplied. Enhancement of the sharp 
signal, which is expressed as the ratio of the input signal IS to the 

25 unsharp signal US, is equivalent to calculating the difference between 
the input signal IS and the unsharp signal US in logarithmic space. 
That is, visual processing that is suited to the logarithmic visual 
characteristics of humans is achieved, 
(ii) 
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The amount of enhancement by the enhancement function F5 is 
large if the input signal IS is large (if it is bright) and is small if the 
input signal IS is small (if it is dark). Also, the amount of enhancement 
toward increased brightness is greater than the amount of enhancement 
5 toward increased darkness. Thus, it is possible to achieve visual 
processing that is suited for our visual characteristics, and visual 
processing that is balanced and natural is achieved, 
(iii) 

It should be noted that if the value C of a particular element of 
10 the profile data obtained by Eq. M3 is C>255, then the value C of that 
element can be regarded as 255. 
(iv) 

With the processing employing the equation M3, although 
compression of the dynamic range is not performed with respect to the 
15 input signal IS, local contrast can be enhanced and the dynamic range 
can be visually compressed and expanded. 

(4) 

«Fourth Profile Data» 
The fourth profile data are determined based on a computation 
20 that includes a function for enhancing the difference between the input 
signal IS and the unsharp signal US in accordance with the value of the 
input signal IS. Due to this, it is for example possible to enhance the 
sharp component of the input signal IS, for example, in accordance with 
the value of the input signal IS. Thus, enhancement of the input signal 
25 IS can be performed suitably over a range including dark areas and light 
areas. 

Further, the fourth profile data are determined based on a 
computation that adds to the enhanced value the value that is obtained 
by compressing the dynamic range of the input signal IS. Thus, it is 
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possible to enhance the sharp component of the input signal IS, for 
example, in accordance with the value of the input signal IS while 
performing compression of the dynamic range. 

This is described in further detail below. 
5 The values C of the elements of the fourth profile data (the values 

of the output signal OS) are expressed by C = F8(A) + F6(A) * F7(A-B) 
(hereinafter, this is referred to as Equation M4) using the value A of the 
input signal IS, the value B of the unsharp signal US, an enhancement 
amount adjustment function F6, an enhancement function F7, and a 

10 dynamic range compression function F8. 

Here, the enhancement amount adjustment function F6 is a 
function that monotonically increases with respect to the value of the 
input signal IS. That is, when the value A of the input signal IS is 
small, the value of the enhancement amount adjustment function F6 also 

15 is small, and when the value A of the input signal IS is large, then the 
value of the enhancement amount adjustment function F6 also is large. 
The enhancement function F7 is any one of the enhancement functions 
Rl to R3 that were described using Fig. 49. The dynamic range 
compression function F8 is the power function described using Fig. 17, 

20 and is expressed by F8(x) = x A y (0<y<l). 

«Equivalent Visual Processing Device 41» 
Fig. 20 shows a visual processing device 41 that is equivalent to 
the visual processing device 1 in which the fourth profile data have been 
registered to the two-dimensional LUT 4. 

25 The visual processing device 41 is a device that outputs an output 

signal OS based on a computation for enhancing the difference between 
the input signal IS and the unsharp signal US in accordance with the 
value of the input signal IS. Due to this, it is for example possible to 
enhance the sharp component of the input signal IS, for example, in 
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accordance with the value of the input signal IS. Thus, enhancement 
can be suitably performed from dark areas through light areas of the 
input signal IS. 

Further, the visual processing device 41 outputs an output signal 
5 OS based on a computation that adds to the enhanced value, the value 
that is obtained by compressing the dynamic range of the input signal IS. 
Thus, it is possible to enhance the sharp component of the input signal 
IS, for example, in accordance with the value of the input signal IS while 
performing compression of the dynamic range. 

10 The visual processing device 41 shown in Fig. 20 is provided with 

a spatial processing portion 42 that performs spatial processing of the 
luminance value of each pixel of an original image that it has obtained as 
an input signal IS and outputs an unsharp signal US, and a visual 
processing portion 43 that uses the input signal IS and the unsharp 

15 signal US to perform visual processing of the original image and outputs 
an output signal OS. 

The spatial processing portion 42 performs the same operation as 
the spatial processing portion 2 of the visual processing device 1, and 
thus it will not be described further. 

20 The visual processing portion 43 is provided with a subtracting 

portion 44 that receives an input signal IS as a first input and an 
unsharp signal US as a second input and then outputs a difference signal 
DS of the difference between the two, an enhancing portion 45 that 
receives the difference signal DS as an input and outputs an enhanced 

25 signal TS, an enhancement amount adjustment portion 46 that receives 
the input signal IS as input and outputs an enhancement amount 
adjusted signal IC, a multiplying portion 47 that receives the 
enhancement amount adjusted signal IC as a first input and the 
enhanced signal TS as a second input and multiplies the enhancement 
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amount adjusted signal IC and the enhanced signal TS to obtain a 
multiplied signal MS that it then outputs, and an output portion 48 that 
receives the input signal IS as a first input and the multiplied signal MS 
as a second input and outputs an output signal OS. The output portion 
5 48 is provided with a DR compression portion 49 that receives the input 
signal IS as input and outputs a DR compressed signal DRS that has 
been subjected to dynamic range (DR) compression, and a summing 
portion 50 that receives the DR compressed signal DRS as a first input 
and the multiplied signal MS as a second input and outputs an output 

10 signal OS. 

«Action of the Equivalent Visual Processing Device 41» 
The operation of visual processing portion 43 is described in 
further detail below. 

The subtracting portion 44 calculates the difference between the 

15 input signal IS, which has the value A, and the unsharp signal US, 
which has the value B, and outputs a difference signal DS having the 
value A-B. The enhancing portion 45 uses the enhancement function 
F7 to output an enhanced signal TS having the value F7(A-B) from the 
difference signal DS, which has the value A-B. The enhancement 

20 amount adjustment portion 46 uses the enhancement amount 
adjustment function F6 to output an enhancement amount adjusted 
signal IC having the value F6(A) from the input signal IS, which has the 
value A. The multiplying portion 47 multiplies the enhancement 
amount adjusted signal IC, which has the value F6(A), and the enhanced 

25 signal TS, which has the value F7(A-B), and outputs the result as a 
multiplied signal MS having the value F6(A) * F7(A-B). The DR 
compression portion 49 uses the dynamic range compression function F8 
to output a DR compressed signal DRS having the value F8(A) from the 
input signal IS, which has the value A. The summing portion 50 takes 
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the sum of the DR compressed signal DRS and the multiplied signal MS, 
which has the value F6(A) * F7(A-B), and outputs an output signal OS 
having the value F8(A) + F6(A) * F7(A-B). 

It should be noted that the calculations using the enhancement 
5 amount adjustment function F6, the enhancement function F7, and the 
dynamic range compression function F8 can each be performed using a 
one- dimensional LUT for that function, or can be performed without 
using a LUT. 

«Effects» 

10 The visual processing device 41 and the visual processing device 

1 provided with the fourth profile data achieve the same visual 
processing effects, 
(i) 

Adjustment of the enhancement amount of the difference signal 
15 DS is performed based on the value A of the input signal IS. Thus, it is 
possible to perform dynamic range compression while maintaining local 
contrast over dark areas to light areas, 
(ii) 

The enhancement amount adjustment function F6 is a function in 
20 which the values increases monotonically and thus it can be a function 
with which the amount of increase in the values of the function 
decreases as the value A of the input signal IS increases. In this case, 
the value of the output signal OS is kept from becoming saturated, 
(iii) 

25 If the enhancement function F7 is the enhancement function R2 

that was described using Fig. 49, then it is possible to inhibit the amount 
of enhancement when the absolute value of the difference signal DS is 
large. Thus, the enhancement value in high-sharpness areas is kept 
from becoming saturated, and this allows visual processing that leads to 
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a visually natural image to be executed. 
«Modified Examples» 
(i) 

With the visual processing device 41, the visual processing 
5 portion 43 can also perform the computation of Eq. M4 based on the 
input signal IS and the unsharp signal US without using the 
two-dimensional LUT 4. In this case, it can also use a one-dimensional 
LUT in the calculations of any of the functions F6 to F8. 
(ii) 

10 If the enhancement function F7 is a direct proportion function of 

a proportion coefficient 1, then there is no particular need to provide the 
enhancing portion 45. 
(iii) 

It should be noted that if the value C of a particular element of 
15 the profile data found through Eq. M4 is outside the range 0<C<255, 
then the value C of that element can be regarded as 0 or 255. 

(5) 

<Fifth Profile Data> 
The fifth profile data are determined based on a computation that 
20 includes a function for enhancing the difference between the input signal 
IS and the unsharp signal US in accordance with the value of the input 
signal IS. Due to this, it is for example possible to enhance the sharp 
component of the input signal IS, for example, in accordance with the 
value of the input signal IS. Thus, enhancement of the input signal IS 
25 can be performed suitably across dark areas to light areas. 
This is described in further detail below. 

It is also possible that in Eq. M4 of the fourth profile data, the 
dynamic range compression function F8 is a direct proportion function of 
a proportion coefficient 1. In this case, the values C of the elements of 



67 



the fifth profile data (the values of the output signal OS) are expressed 
by C = A + F6(A) * F7(A-B) (hereinafter, this is referred to as Equation 
M5) using the value A of the input signal IS, the value B of the unsharp 
signal US, the enhancement amount adjustment function F6, and the 
5 enhancement function F7. 

«Equivalent Visual Processing Device 51» 
Fig. 21 shows a visual processing device 51 that is equivalent to 
the visual processing device 1 in which the fifth profile data have been 
registered to the two-dimensional LUT 4. 

10 The visual processing device 51 is a device that outputs an output 

signal OS based on a computation for enhancing the difference between 
the input signal IS and the unsharp signal US in accordance with the 
value of the input signal IS. Due to this, it is for example possible to 
enhance the sharp component of the input signal IS, for example, in 

15 accordance with the value of the input signal IS. Thus, enhancement of 
the input signal IS can be suitably performed over a range that spans 
dark areas and light areas. 

The visual processing device 51 shown in Fig. 21 differs from the 
visual processing device 41 shown in Fig. 20 in that it is not provided 

20 with the DR compression portion 49. The visual processing device 51 
shown in Fig. 21 is described below, assigning the same reference 
numerals to portions that perform the same operations as those of the 
visual processing portion 41 shown in Fig. 20, and detailed description 
thereof is omitted. 

25 The visual processing device 51 is provided with a spatial 

processing portion 42 that executes spatial processing of the brightness 
value of each pixel of an original image that it has obtained as an input 
signal IS and outputs the result as an unsharp signal US, and a visual 
processing portion 52 that uses the input signal IS and the unsharp 
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signal US to perform visual processing of the original image and outputs 
the result as an output signal OS. 

The spatial processing portion 42 performs the same operation as 
the spatial processing portion 2 of the visual processing device 1, and 
5 thus it will not be described further. 

The visual processing portion 52 is provided with a subtracting 
portion 44 that receives an input signal IS as a first input and an 
unsharp signal US as a second input and then outputs a difference signal 
DS of the difference between the two, an enhancing portion 45 that 
10 receives the difference signal DS as input and outputs an enhanced 
signal TS, an enhancement amount adjustment portion 46 that receives 
the input signal IS as input and outputs an enhancement amount 
adjusted signal IC, a multiplying portion 47 that receives the 
enhancement amount adjusted signal IC as a first input and the 
15 enhanced signal TS as a second input and multiplies the enhancement 
amount adjusted signal IC and the enhanced signal TS to obtain a 
multiplied signal MS that it then outputs, and a summing portion 53 
that receives the input signal IS as a first input and the multiplied 
signal MS as a second input and outputs an output signal OS. 
20 «Action of the Equivalent Visual Processing Device 51» 

The operation of the visual processing portion 52 is described in 
further detail below. 

The subtracting portion 44, the enhancing portion 45, the 
enhancement amount adjustment portion 46, and the multiplying 
25 portion 47 perform the same operations as those described with regard to 
the visual processing device 41 shown in Fig. 20. 

The summing portion 53 takes the sum of the input signal IS, 
which has the value A, and the multiplied signal MS, which has the 
value F6(A) * F7(A-B), and outputs an output signal OS having the 
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value A + F6(A) * F7(A-B). 

It should be noted that as described with respect to the visual 
processing device 41 shown in Fig. 20, the calculations using the 
enhancement amount adjustment function F6 and the enhancement 
5 function F7 can each be performed using a one-dimensional LUT for that 
function, or can be performed without using a LUT. 
«Effects» 

The visual processing device 51 and the visual processing device 
1 provided with the fifth profile data achieve the same visual processing 
10 effects. Moreover, they achieve substantially the same visual processing 
effects as those attained by the visual processing device 41 and the 
visual processing device 1 provided with the fourth profile data, 
(i) 

The adjustment of the enhancement amount of the difference 
15 signal DS is performed based on the value A of the input signal IS. 
Thus, it is possible to attain a uniform amount of contrast enhancement 
from dark areas up to light areas. 

«Modified Examples» 
(i) 

20 If the enhancement function F7 is a direct proportion function of 

a proportion coefficient 1, then it is not particularly necessary to provide 
the enhancing portion 45. 
(ii) 

It should be noted that if the value C of a particular element of 
25 the profile data found through Eq. M5 is outside the range 0<C<255, 
then the value C of that element can be taken as 0 or 255. 

(6) 

<Sixth Profile Data» 
The sixth profile data are determined based on a computation 
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that corrects the gradation of a value that is obtained by adding the 
value of the input signal IS to a value obtained by enhancing the 
difference between the input signal IS and the unsharp signal US. 
Thus, it is for example possible to achieve visual processing in which 
5 gradation correction is performed with respect to an input signal IS 
whose sharp component has been enhanced. 

This is described in further detail below. 

The values C of the elements of the sixth profile data (the values 
of the output signal OS) are expressed by C = F10(A + F9(A-B)) 

10 (hereinafter, this is referred to as Equation M6) using the value A of the 
input signal IS, the value B of the unsharp signal US, an enhancement 
function F9, and a gradation correction function F10. 

Here, the enhancement function F9 is any one of the 
enhancement functions Rl to R3 that were described using Fig. 49. The 

15 gradation correction function F10 is a function that is used in ordinary 
gradation correction, and for example is a gamma correction function, an 
S-shaped gradation correction function, or an inverted S-shaped 
gradation correction function. 

«Equivalent Visual Processing Device 61» 

20 Fig. 22 shows a visual processing device 61 that is equivalent to 

the visual processing device 1 in which the sixth profile data have been 
registered to the two-dimensional LUT 4. 

The visual processing device 61 is a device that outputs an output 
signal OS based on a computation that corrects the gradation of the 

25 value obtained by adding the value of the input signal IS to a value that 
is obtained by enhancing the difference between the input signal IS and 
the unsharp signal US. Thus, it is for example possible to achieve 
visual processing in which gradation correction is performed with respect 
to an input signal IS whose sharp component has been enhanced. 
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The visual processing device 61 shown in Fig. 22 is provided with 
a spatial processing portion 62 that performs spatial processing of the 
luminance value of each pixel of an original image that it has obtained as 
an input signal IS and outputs an unsharp signal US, and a visual 
5 processing portion 63 that uses the input signal IS and the unsharp 
signal US to perform visual processing of the original image and outputs 
an output signal OS. 

The spatial processing portion 62 performs the same operation as 
the spatial processing portion 2 of the visual processing device 1, and 
10 thus it will not be described further. 

The visual processing portion 63 is provided with a subtracting 
portion 64 that receives an input signal IS as a first input and an 
unsharp signal US as a second input and then outputs a difference signal 
DS of the difference between the two, an enhancing portion 65 that 
15 receives the difference signal DS as input and outputs an enhanced 
signal TS that has been enhanced, a summing portion 66 that receives 
the input signal IS as a first input and the enhanced signal TS as a 
second input and outputs a summed signal PS that is the sum of the two, 
and a gradation correction portion 67 that receives the summed signal 
20 PS as input and outputs an output signal OS. 

«Action of the Equivalent Visual Processing Device 61» 
The operation of the visual processing portion 63 is described in 
further detail below. 

The subtracting portion 64 calculates the difference between the 
25 input signal IS, which has the value A, and the unsharp signal US, 
which has the signal B, and outputs a difference signal DS having the 
value A-B. The enhancing portion 65 uses the enhancement function 
F9 to output an enhanced signal TS having the value F9(A-B) from the 
difference signal DS, which has the value A-B. The summing portion 
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66 takes the sum of the input signal IS, which has the value A, and the 
enhanced signal TS, which has the value F9(A-B), and outputs a 
summed signal PS having the value A + F9(A-B). The gradation 
correction portion 67 uses the gradation correction function F10 to 
5 output an output signal OS having the value F10(A + F9(A-B)) from the 
summed signal PS, which has the value A + F9(A-B). 

It should be noted that the calculations using the enhancement 
function F9 and the gradation correction function F10 can each be 
performed using a one-dimensional LUT for that function, or can be 
10 performed without using a LUT. 
«Effects» 

The visual processing device 61 and the visual processing device 
1 furnished with the sixth profile data achieve the same visual 
processing effects. 

15 (i) 

The difference signal DS is enhanced by the enhancement 
function F9 and is added to the input signal IS. Thus, it becomes 
possible to enhance the contrast of the input signal IS. Moreover, the 
gradation correction portion 67 corrects the gradation of the summed 

20 signal PS. Thus, it is for example possible to further enhance the 
contrast at intermediate tones, which frequently appear in original 
images. It is also possible to brighten the entire summed signal PS, for 
example. Thus, spatial processing and gradation processing can be 
simultaneously achieved together. 

25 «Modified Examples» 

(i) 

With the visual processing device 61, the visual processing 
portion 63 can also perform the computation of Eq. M6 based on the 
input signal IS and the unsharp signal US without using the 
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two-dimensional LUT 4. In this case, it is also possible to use a 
one-dimensional LUT for each of the calculations of the functions F9 and 
F10. 

(ii) 

5 It should be noted that if the value C of a particular element of 

the profile data found through Eq. M6 is outside the range 0<C<255, 
then the value C of that element can be regarded as 0 or 255. 
(7) 

«Seventh Profile Data» 

10 The seventh profile data are determined based on a computation 

that adds a value that is obtained by correcting the gradation of the 
input signal IS to a value that is obtained by enhancing the difference 
between the input signal IS and the unsharp signal US. Here, 
enhancement of the sharp component and gradation correction of the 

15 input signal IS are performed independently. Thus, it is possible to 
perform a fixed enhancement of the sharp component, regardless of the 
amount of gradation correction of the input signal IS. 
This is described in further detail below. 

The values C of the elements of the seventh profile data (the 
20 values of the output signal OS) are expressed by C = F12(A) + Fll(A-B) 
(hereinafter, this is referred to as Equation M7) using the value A of the 
input signal IS, the value B of the unsharp signal US, an enhancement 
function Fll, and a gradation correction function F12. 

Here, the enhancement function Fll is any one of the 
25 enhancement functions Rl to R3 described using Fig. 49. The gradation 
correction function F12 is for example a gamma correction function, an 
S-shaped gradation correction function, or an inverted S-shaped 
gradation correction function. 

«Equivalent Visual Processing Device 71» 
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Fig. 23 shows a visual processing device 71 that is equivalent to 
the visual processing device 1 in which the seventh profile data have 
been registered to the two-dimensional LUT 4. 

The visual processing device 71 is a device that outputs an output 
5 signal OS based on a computation that adds a value that is obtained by 
correcting the gradation of the input signal IS to a value that is obtained 
by enhancing the difference between the input signal IS and the unsharp 
signal US. Here, enhancement of the sharp component and gradation 
correction of the input signal IS are performed independently. Thus, it 
10 is possible to perform a fixed enhancement of the sharp component, 
regardless of the amount of gradation correction of the input signal IS. 

The visual processing device 71 shown in Fig. 23 is provided with 
a spatial processing portion 72 that performs spatial processing of the 
luminance value of each pixel of an original image that it has obtained as 
15 an input signal IS and outputs an unsharp signal US, and a visual 
processing portion 73 that uses the input signal IS and the unsharp 
signal US to perform visual processing of the original image and outputs 
an output signal OS. 

The spatial processing portion 72 performs the same operation as 
20 the spatial processing portion 2 of the visual processing device 1, and 
thus it will not be described further. 

The visual processing portion 73 is provided with a subtracting 
portion 74 that receives the input signal IS as a first input and the 
unsharp signal US as a second input and then outputs a difference signal 
25 DS of the difference between the two, an enhancing portion 75 that 
receives the difference signal DS as input and outputs an enhanced 
signal TS that has been enhanced, a gradation correction portion 76 that 
receives the input signal IS as input and outputs a gradation corrected 
signal GC whose gradation has been corrected, and a summing portion 
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77 that receives the gradation corrected signal GC as a first input and 
the enhanced signal TS as a second input and outputs an output signal 
OS. 

«Action of the Equivalent Visual Processing Device 71» 
5 The operation of the visual processing portion 73 is described in 

further detail below. 

The subtracting portion 74 calculates the difference between the 
input signal IS, which has the value A, and the unsharp signal US, 
which has the signal B, and outputs a difference signal DS having the 
10 value A-B. The enhancing portion 75 uses the enhancement function 
Fll to output an enhanced signal TS having the value F 11 (A-B) from the 
difference signal DS, which has the value A-B. The gradation 
correction portion 76 uses the gradation correction function F12 to 
output a gradation corrected signal GC having the value F12(A) from the 
15 input signal IS having the value A. The summing portion 77 takes the 
sum of the gradation corrected signal GC, which has the value F12(A), 
and the enhanced signal TS, which has the value Fll(A-B), and outputs 
an output signal OS having the value F12(A) + Fll(A-B). 

It should be noted that the calculations using the enhancement 
20 function Fll and the gradation correction function F12 can each be 
performed using a one-dimensional LUT for that function, or can be 
performed without using a LUT. 
«Effects» 

The visual processing device 71 and the visual processing device 
25 1 furnished with the seventh profile data achieve the same visual 
processing effects, 
(i) 

After the input signal IS has been subjected to gradation 
correction by the gradation correction portion 76, it is added to the 
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enhanced signal TS. Thus, by adding that input signal IS to the 
enhanced signal TS, it is possible to emphasize local contrast even in 
regions where there has been little change in the gradation by the 
gradation correction function F12, that is, regions in which the contrast 
5 is lowered. 

«Modified Examples» 

(i) 

With the visual processing device 71, the visual processing 
portion 73 can also perform the computation of Eq. M7 based on the 
10 input signal IS and the unsharp signal US without using the 
two-dimensional LUT 4. In this case, it is also possible to use a 
one-dimensional LUT in the calculations of the functions Fll and F12. 
(ii) 

It should be noted that if the value C of a particular element of 
15 the profile data found through Eq. M7 is outside the range 0<C<255, 
then the value C of that element can be regarded as 0 or 255. 

(8) 

«Modified Examples of the First through Seventh Profile 

Data» 

20 (i) 

In (l) through (7) above, each element of the first through 
seventh profile data was described as storing a value that has been 
calculated based on an equation Ml through M7. It was also explained 
that the value of an element in the profile data that has been calculated 

25 by an equation Ml through M7 can be clipped if the value is outside of 
the range of values that can be stored as profile data. 

Moreover, some of the values of the profile data can be any value. 
For example, if the value of the input signal IS is large but the value of 
the unsharp signal US is small, such as small bright areas in a dark 
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nighttime setting (such as areas of neon in a nighttime setting), then the 
value of the input signal IS that has been subjected to visual processing 
will have little effect on the picture quality. Thus, for portions where 
the value after visual processing will have little effect on the picture 
5 quality, the values stored in the profile data can be approximate values 
of the value calculated by the equations Ml through M7, or can be 
arbitrary values. 

Even if the values stored in the profile data are approximate 
values of the value calculated by the equations Ml through M7, or are 

10 arbitrary values, it is preferable that the values stored for the same 
values of the input signal IS and the unsharp signal US maintain the 
monotonically increasing, or monotonically decreasing, relationship with 
respect to the values of the input signal IS and the unsharp signal US. 
In the profile data created based on the equations Ml to M7, the values 

15 stored by the profile data with respect to the same values for the input 
signal IS and the unsharp signal US show an overview of the 
characteristics of the profile data. Thus, to maintain the characteristics 
of the two-dimensional LUT, it is preferable that the above relationship 
is maintained when tuning the profile data. 

20 

[Second Embodiment] 

A visual processing device 600 serving as a second embodiment of 
the present invention is described using Figs. 24 through 39. 

The visual processing device 600 is a visual processing device for 
25 performing visual processing of an image signal (input signal IS) and 
outputting a visually processed image (output signal OS), and it 
performs visual processing that is in accordance with the environment in 
which a display device (not shown) for displaying the output signal OS is 
located (hereinafter, this is referred to as the display environment). 
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More specifically, the visual processing device 600 is a device that 
fixes drops in the "visual contrast" of the display image due to the effect 
of ambient light of the display environment by performing visual 
processing that utilizes our visual characteristics. 
5 The visual processing device 600, together with a device that 

performs color processing of an image signal, constitutes an image 
processing device in a machine that handles images, such as a computer, 
a television, a digital camera, a portable telephone, a PDA, a printer, or a 
scanner. 

10 <Visual Processing Device 600> 

Fig. 24 shows the basic structure of the visual processing device 

600. 

The visual processing device 600 comprises a target contrast 
transformation portion 601, a transformed signal processing portion 602, 
15 an actual contrast transformation portion 603, a target contrast setting 
portion 604, and an actual contrast setting portion 605. 

The target contrast transformation portion 601 receives the input 
signal IS as a first input and a target contrast Cl that has been set by 
the target contrast setting portion 604 as a second input, and outputs a 
20 target contrast signal JS. It should be noted that the target contrast Cl 
will be defined later. 

The transformed signal processing portion 602 receives the target 
contrast signal JS as a first input, the target contrast Cl as a second 
input, and the actual contrast C2 that has been set by the actual 
25 contrast setting portion 605 as a third input, and outputs a visually 
processed signal KS, which is the target contrast signal JS after being 
subjected to visual processing. It should be noted that the actual 
contrast C2 will be defined later. 

The actual contrast transformation portion 603 receives the 
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visually processed signal KS as a first input and the actual contrast C2 
as a second input, and outputs an output signal OS. 

The user is allowed to set the values of the target contrast Cl and 
the actual contrast C2 with the target contrast setting portion 604 and 
5 the actual contrast setting portion 605 via an input interface, for 
example. 

These portions are described in greater detail below. 

<Target Contrast Transformation Portion 60 1> 

The target contrast transformation portion 601 transforms the 
10 input signal IS that has been input to the visual processing portion 600 
into a target contrast signal JS that is suited for contrast expression. 
Here, the input signal IS is expressed by a gradation in which the 
luminance of the image that has been input to the visual processing 
portion 600 is from [0.1 to 1.0]. 
15 The target contrast transformation portion 601 uses a target 

contrast Cl (value [m]) to transform the input signal IS (value [P]) 
through "Eq. M20" and outputs the target contrast signal JS (value [A]). 
Here, Eq. M20 is A = {(m-l)/m} * P+l/m. 

The value [m] of the target contrast Cl is set as the contrast 
20 value at which the displayed image that is displayed by the display 
device can be seen at the best contrast. 

Here, the contrast value is a value that is expressed as the 
brightness ratio of the white level to the black level of the image, and 
indicates the luminance value of the white level in a case where the 
25 black level has been set to 1 (black level = white level = l=m). 

The value [m] of the target contrast Cl is preferably set to 
approximately 100 to 1000 (black level = white level = 1 = 100 to 1=1000), 
but this can also be determined based on the brightness ratio of the 
white level to the black level that can be displayed by the display device. 
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Transformation by Equation M20 is described in further detail 
using Fig. 25. Fig. 25 is a graph that shows the relationship between 
the value of the input signal IS (horizontal axis) and the value of the 
target contrast signal JS (vertical axis). As shown in Fig. 25, an input 
5 signal IS whose value is in the range of [0.0 to 1.0] is transformed into a 
target contrast signal JS having a value in the range of [l/m to 1.0] by 
the target contrast transformation portion 601. 

<Transformed Signal Processing Portion 602> 

Using Fig. 24, the transformed signal processing portion 602 is 

10 described in greater detail below. 

The transformed signal processing portion 602 compresses the 
dynamic range while maintaining local contrast in the target contrast 
signal JS that is input, and outputs a visually processed signal KS. 
More specifically, the transformed signal processing portion 602 has the 

15 same structure, action, and effect as if in the visual processing device 21 
described in the first embodiment the input signal IS (see Fig. 16) is 
regarded as the target contrast signal JS and the output signal OS (see 
Fig. 16) is regarded as the visually processed signal KS. 

The transformed signal processing portion 602 outputs the 

20 visually processed signal KS based on a computation that enhances the 
ratio between the target contrast signal JS and the unsharp signal US. 
Thus, it is for example possible to achieve the visual processing for 
enhancing the sharp component. 

Further, the transformed signal processing portion 602 outputs 

25 the visually processed signal KS based on a computation for compressing 
the dynamic range of the ratio between the target contrast signal JS that 
has been enhanced and the unsharp signal US. Thus, it is possible to 
achieve visual processing in which the sharp component is enhanced 
while the dynamic range is compressed, for example. 
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«Structure of the Transformed Signal Processing Portion 

602» 

The transformed signal processing portion 602 is provided with a 
spatial processing portion 622 that performs spatial processing on the 
5 luminance value of each pixel in the target contrast signal JS and 
outputs an unsharp signal US, and a visual processing portion 623 that 
uses the target contrast signal JS and the unsharp signal US to perform 
visual processing of the target contrast signal JS and outputs a visually 
processed signal KS. 

10 The spatial processing portion 622 performs the same operation 

as the spatial processing portion 2 of the visual processing device 1 (see 
Fig. l), and thus detailed description thereof is omitted. 

The visual processing portion 623 is provided with a dividing 
portion 625, an enhancing portion 626, and an output portion 627 that 

15 has a DR compression portion 628 and a multiplying portion 629. 

The dividing portion 625 receives the target contrast signal JS as 
a first input and the unsharp signal US as a second input and divides 
the target contrast signal JS by the unsharp signal US to obtain a 
divided signal RS that it then outputs. The enhancing portion 626 

20 receives the divided signal RS as a first input, the target contrast Cl as 
a second input, and the actual contrast C2 as a third input, and outputs 
an enhanced signal TS. 

The output portion 627 receives the target contrast signal JS as a 
first input, the enhanced signal TS as a second input, the target contrast 

25 Cl as a third input, and the actual contrast C2 as a fourth input, and 
outputs a visually processed signal KS. The DR compression portion 
628 receives the target contrast signal JS as a first input, the target 
contrast Cl as a second input, and the actual contrast C2 as a third 
input, and outputs a DR compressed signal DRS whose dynamic range 
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(DR) has been compressed. The multiplying portion 629 receives the 
DR compressed signal DRS as a first input and the enhanced signal TS 
as a second input, and outputs the visually processed signal KS. 

«Action of the Transformed Signal Processing Portion 602» 
5 The transformed signal processing portion 602 uses the target 

contrast Cl (value [m]) and the actual contrast C2 (value [n]) to 
transform the target contrast signal JS (value [A]) through "Eq. M2" and 
outputs the result as a visually processed signal KS (value [C]). Here, 
Eq. M2 is expressed as C = F4(A) * F5(A/B) using the dynamic range 

10 compression function F4 and the enhancement function F5. It should be 
noted that the value [B] is the value of the unsharp signal US obtained 
by subjecting the target contrast signal JS to spatial processing. 

The dynamic range compression function F4 is a "power function" 
that is a concave monotonically increasing function, and is expressed by 

15 F4(x) = x A y. The exponent y of the dynamic range compression function 
F4 is expressed using normal logarithms by y=log(n)/log(m). The 
enhancement function F5 is a power function, and is expressed by F5(x) 
= x A (l-y). 

The relationship between Eq. M2 and the operations of the 
20 various portions of the transformed signal processing portion 602 is 
described in further detail below. 

The spatial processing portion 622 performs spatial processing 
with respect to the target contrast signal JS, which has the value [A], 
and outputs the result as an unsharp signal US having the value [B]. 
25 The dividing portion 625 divides the target contrast signal JS, 

which has the value [A], by the unsharp signal US, which has the value 
[B], and outputs the result as the divided signal RS, which has the value 
[A/B]. The enhancing portion 626 uses the enhancement function F5 to 
obtain an enhanced signal TS having the value [F5(A/B)] from the 
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divided signal RS, which has the value [A/B], and outputs this enhanced 
signal TS. The DR compression portion 628 uses the dynamic range 
compression function F4 to obtain a DR compressed signal DRS having 
the value [F4(A)] from the target contrast signal JS, which has the value 
5 [A], and outputs the DR compressed signal DRS. The multiplying 
portion 629 multiplies the DR compressed signal DRS, which has the 
value [F4(A)], by the enhanced signal TS, which has the value [F5(A/B)], 
and outputs the result as a visually processed signal KS having the 
value [F4(A) * F5(A/B)]. 

10 It should be noted that the calculations using the dynamic range 

compression function F4 and the enhancement function F5 can each be 
performed using a one-dimensional LUT for that function, or can be 
performed without using a LUT. 

«Effect of the Transformed Signal Processing Portion 602» 

15 The visual dynamic range in the visually processed signal KS is 

determined by the value of the dynamic range compression function F4. 

The transformation by Eq. M2 is described in more specific detail 
using Fig. 26. Fig. 26 is a graph showing the relationship between the 
value of the target contrast signal JS (horizontal axis) and the value 

20 obtained by applying the dynamic range compression function F4 to the 
target contrast signal JS (vertical axis). As shown in Fig. 26, the 
dynamic range of the target contrast signal JS is compressed by the 
dynamic range compression function F4. More specifically, a target 
contrast signal JS in the value range of [l/m to 1.0] is transformed by the 

25 dynamic range compression function F4 to a value range of [l/n to 1.0]. 
As a result, the viewable dynamic range of the visually processed signal 
KS is compressed to l/n (minimum value : maximum value = l=n). 

The actual contrast C2 shall be described here. The value [n] of 
the actual contrast C2 is set as the viewable contrast value of the 
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displayed image under the circumstances of the ambient light of the 
display environment. That is, the value [n] of the actual contrast C2 
can be determined to be the value obtained by lowering the value [m] of 
the target contrast Cl by an amount that corresponds to the impact of 
5 the luminance of the ambient light of the display environment. 

As the result of using the value [n] of the actual contrast C2 that 
has been set in this way, the dynamic range of the target contrast signal 
JS following from Eq. M2 is compressed to between i:m and i:n. It 
should be noted that here "dynamic range" means the ratio between the 

10 minimum value and the maximum value of the signal. 

On the other hand, the change in the local contrast in the 
visually processed signal KS is expressed as the ratio of the change 
amount before and after transformation between the value [A] of the 
target contrast signal JS and the value [C] of the visually processed 

15 signal KS. Here, the value [B] of the unsharp signal US in a local, that 
is, narrow range is regarded as constant. Thus, the ratio between the 
amount of change in the value [C] and the amount of change in the value 
[A] in Eq. M2 is 1, meaning that the local contrast does not change 
between the target contrast signal JS and the visually processed signal 

20 KS. 

Human vision has the characteristic that as long as local contrast 
is maintained, an image will be viewed at the same contrast even if there 
is a drop in the overall contrast. Thus, with the transformed signal 
processing portion 602, it is possible to achieve visual processing with 
25 which the dynamic range of the target contrast signal JS is compressed 
but the viewable contrast is not lowered. 

<Actual Contrast Transformation Portion 603> 
The actual contrast transformation portion 603 is described in 
detail using Fig. 24. 
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The actual contrast transformation portion 603 transforms the 
visually processed signal KS into image data of a range that can be input 
to a display device (not shown). The image data of a range that can be 
input to a display device are image data in which the luminance values 
5 of the image are expressed in a gray scale value range of [0.0 to 1.0] . 

The actual contrast transformation portion 603 uses the actual 
contrast C2 (value [n]) to transform the visually processed signal KS 
(value [C]) with "Equation M21," and outputs the result as an output 
signal OS (value [Q]). Here, Eq. M21 is Q = {n/(n-l)} * C - {l/(n-l)}. 

10 This transformation following from Eq. M21 is described in 

greater detail using Fig. 27. Fig. 27 is a graph showing the relationship 
between the value of the visually processed signal KS (horizontal axis) 
and the value of the output signal OS (vertical axis). As shown in Fig. 
27, the actual contrast transformation portion 603 transforms a visually 

15 processed signal KS, which is in the range of [l/n to 1.0], into an output 
signal OS that has a range of [0.0 to 1.0]. Here, the value of the output 
signal OS decreases with respect to the value of the visually processed 
signal KS. The amount of this decrease corresponds to the impact of the 
ambient light on the luminance values of the display image. 

20 It should be noted that if a visually processed signal KS having a 

value less than [l/n] is input, then the actual contrast transformation 
portion 603 transforms the output signal OS to the value [0]. Further, if 
a visually processed signal KS having a value greater than [l] is input, 
then the actual contrast transformation portion 603 transforms the 

25 output signal OS to the value [l]. 

<Effects of the Visual Processing Device 600> 

The visual processing device 600 attains the same effects as the 
visual processing device 21 described in the first embodiment. Those 
characteristic effects of the visual processing device 600 are discussed 
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below. 

(i) 

When there is ambient light in the display environment in which 
the visual processing device 600 displays the output signal OS, the 
5 ambient light affects how the output signal OS is viewed. However, the 
output signal OS is a signal in which processing to correct the impact of 
ambient light has been executed by the actual contrast transformation 
portion 603. That is, in a display environment where there is ambient 
light, the output signal OS that is displayed by the display device is 

10 viewed as a display image having the characteristics of the visually 
processed signal KS. 

As for the characteristics of the visually processed signal KS, like 
for example the output signal OS of the visual processing device 21 
described in the first embodiment (see Fig. 16), the local contrast is 

15 maintained while compressing the dynamic range of the entire image. 
That is, the visually processed signal KS is a signal in which the target 
contrast Cl at which locally the display image is displayed most suitably 
is maintained while the dynamic range is compressed to a dynamic range 
that can be displayed in the presence of the ambient light (this 

20 corresponds to the actual contrast C2). 

Thus, with the visual processing device 600, it is possible to 
correct the contrast, which drops due to the presence of ambient light, 
while maintaining the visual contrast through processing that exploits 
the viewing characteristics. 

25 <Visual Processing Method> 

The visual processing method for achieving the same effects as 
the visual processing device 600 is described below using Fig. 28. It 
should be noted that the specific processing of each step is the same as 
that in the visual processing device 600, and thus description thereof will 
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be omitted. 

With the visual processing method shown in Fig. 28, first the 
target contrast Cl that has been set and the actual contrast C2 are 
obtained (S60l). Next, the target contrast Cl that has been obtained is 
5 used to transform the input signal IS (step S602), and the result is 
output as a target contrast signal JS. The target contrast signal JS is 
then subjected to spatial processing (step S603) and the result is output 
as an unsharp signal US. The target contrast signal JS is then divided 
by the unsharp signal US (step S604) and the result is output as a 

10 divided signal RS. The divided signal RS is enhanced by the 
enhancement function F5, which is a "power function" whose exponent is 
determined by the target contrast Cl and the actual contrast C2 (step 
S605), and the result is output as an enhanced signal TS. On the other 
hand, the dynamic range of the target contrast signal JS is compressed 

15 by the dynamic range compression function F4, which is a "power 
function" whose exponent is determined by the target contrast Cl and 
the actual contrast C2 (step S606), and the result is output as a DR 
compressed signal DRS. Next, the enhanced signal TS output in step 
S605 and the DR compressed signal DRS output in step S606 are 

20 multiplied (step S607), and the result is output as a visually processed 
signal KS. Next, the actual contrast C2 is used to transform the 
visually processed signal KS (step S608), and the result is output as an 
output signal OS. The processes of steps S602 through S608 are 
repeated for every pixel of the input signal IS (step S609). 

25 The steps of the visual processing method shown in Fig. 28 can 

also be achieved as a visual processing program in the visual processing 
device 600 or another computer, for example. It is also possible for all 
the processes of steps S604 to S607 to be executed at one time by 
performing the calculation of Eq. M2. 
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<Modified Examples> 

The present invention is not limited to the foregoing embodiment, 
and various modifications and improvements thereof are possible 
without departing from the gist of the invention. 
5 (i) Eq. M2 - A Case in Which the Enhancement Function F5 Is 

Not Provided - 

In the foregoing embodiment, the transformed signal processing 
portion 602 was described as outputting a visually processed signal KS 
based on Eq. M2. Here, it is also possible for the transformed signal 

10 processing portion 602 to output a visually processed signal KS based 
only on the dynamic range enhancement function F4. In this case, it is 
not necessary for the transformed signal processing portion 602 of this 
modified example to be provided with the spatial processing portion 622, 
the dividing portion 625, the enhancing portion 626, or the multiplying 

15 portion 629, and it is only necessary that it is provided with the DR 
compression portion 628. 

With the transformed signal processing portion 602 of this 
modified example, it is possible to output a visually processed signal KS 
that has been compressed to a dynamic range that can be displayed 

20 under the conditions of ambient light. 

(ii) Enhancement Function F5 - Exponents and Other Modified 
Examples - 

In the foregoing embodiment, the enhancement function F5 is a 
"power function" and expressed as F5(x) = x A (l-y). Here, the exponent 
25 of the enhancement function F5 can be a function of the value [A] of the 
target contrast signal JS or the value [B] of the unsharp signal US. 
Specific Examples «1» to «6» are described below. 
«1» 

If the exponent of the enhancement function F5 is a function of 
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the value [A] of the target contrast signal JS, and the value [A] of the 
target contrast signal JS is larger than the value [B] of the unsharp 
signal US, then the enhancement function F5 is a function that 
monotonically decreases. More specifically, the exponent of the 
5 enhancement function F5 is expressed as al(A) * (l-y), where the 
function al(A) is a function that monotonically decreases with respect to 
the value [A] of the target contrast signal JS as shown in Fig. 29. It 
should be noted that the maximum value of the function al(A) is [l.O]. 

In this case, there is little enhancement of the local contrast of 

10 high luminance portions by the enhancement function F5. For this 
reason, if the luminance of a target pixel is higher than the luminance of 
the surrounding pixels, then the local contrast of the high luminance 
portion is kept from being over-enhanced. That is, the luminance of the 
target pixel is kept from becoming saturated at the high luminance, 

15 resulting in "blooming." 
«2» 

If the exponent of the enhancement function F5 is a function of 
the value [A] of the target contrast signal JS, and the value [A] of the 
target contrast signal JS is smaller than the value [B] of the unsharp 

20 signal US, then the enhancement function F5 is a function that 
monotonically increases. More specifically, the exponent of the 
enhancement function F5 is expressed as a2(A) * (l-y), where the 
function a2(A) is a function that monotonically increases with respect to 
the value [A] of the target contrast signal JS is shown in Fig. 30. It 

25 should be noted that the maximum value of the function a2(A) is [l.O]. 

In this case, there is little enhancement of the local contrast of 
low luminance portions by the enhancement function F5. For this 
reason, if the luminance of a target pixel is lower than the luminance of 
the surrounding pixels, then the local contrast of the low luminance 
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portion is kept from being over-enhanced. That is, the luminance of the 
target pixel is kept from being saturated at the low luminance and 
resulting in black squashing. 
«3» 

5 If the exponent of the enhancement function F5 is a function of 

the value [A] of the target contrast signal JS, and the value [A] of the 
target contrast signal JS is larger than the value [B] of the unsharp 
signal US, then the enhancement function F5 is a function that 
monotonically increases. More specifically, the exponent of the 

10 enhancement function F5 is expressed as ct3(A) * (l-y), where the 
function a3(A) is a function that monotonically increases with respect to 
the value [A] of the target contrast signal JS as shown in Fig. 31. It 
should be noted that the maximum value of the function a3(A) is [l.O]. 

In this case, there is little enhancement of the local contrast of 

15 low luminance portions by the enhancement function F5. For this 
reason, if the luminance of a target pixel is higher than the luminance of 
the surrounding pixels, then the local contrast of the low luminance 
portion is kept from being over-enhanced. The low luminance portions 
in the image have a small signal level, and thus the ratio to noise is 

20 relatively high, but by performing this processing it is possible to keep 
the SN ratio from becoming worse. 
«4» 

The exponent of the enhancement function F5 is a function of the 
value [A] of the target contrast signal JS and the value [B] of the 
25 unsharp signal US, and monotonically decreases with respect to the 
absolute value of the difference between the value [A] and the value [B]. 
In other words, the exponent of the enhancement function F5 could be 
said to be a function that increases the closer the ratio of the value [A] to 
the value [B] is to 1. More specifically, the exponent of the 
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enhancement function F5 is expressed as a4(A,B) * (l _ y), where the 
function a4(A,B) is a function that monotonically decreases with respect 
to the absolute value of the value [A-B] as shown in Fig. 32. 

In this case, it is possible to particularly enhance the local 
5 contrast in target pixels whose difference in brightness with respect to 
the surrounding pixels is small, and to inhibit the enhancement of the 
local contrast in target pixels whose difference in brightness with respect 
to the surrounding pixels is large. 
«5» 

10 It is also possible to set an upper limit or a lower limit to the 

result of the computation of the enhancement function F5 in «1» 
though «4» above. More specifically, if the value [F5(A/B)] is greater 
than a predetermined upper limit value, then the predetermined upper 
limit value can be adopted as the result of the computation of the 

15 enhancement function F5. Likewise, if the value [F5(A/B)] is lower than 
a predetermined lower limit value, then the predetermined lower limit 
value can be adopted as the result of the computation of the 
enhancement function F5. 

In this case, it is possible to restrict the amount that the local 

20 contrast is enhanced by the enhancement function F5 to a suitable range, 
and this inhibits over- or under-enhancement of the contrast. 
«6» 

It should be noted that «1» through «5» discussed above can 
be similarly adopted in a case where a computation that employs the 
25 enhancement function F5 is performed in the first embodiment (for 
example, in <Profile Data> (2) or (3) in the first embodiment). It should 
be noted that in the first embodiment the value [A] is the value of the 
input signal IS, and the value [B] is the value of the unsharp signal US 
that is obtained by spatially processing the input signal IS. 
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(iii) Equation M2 - Case in which Dynamic Range Compression is 
not Performed - 

In the foregoing embodiment, the transformed signal processing 
portion 602 was described as having the same configuration as the visual 
5 processing device 21 shown in the first embodiment. Here, as a 
modified example of the transformed signal processing portion 602 it is 
possible for it to have the same configuration as the visual processing 
device 31 shown in the first embodiment (see Fig. 19). More specifically, 
it is possible to achieve a modified example of the transformed signal 

10 processing portion 602 by taking the visual processing device 31 and 
regarding the input signal IS as the target contrast signal JS and 
regarding the output signal OS as the visually processed signal KS. 

In this case, with the transformed signal processing portion 602 
of this modified example, the visually processed signal KS (value [C]) is 

15 output based on "Equation M3" with respect to the target contrast signal 
JS (value [A]) and the unsharp signal US (value [B]). Here, Eq. M3 is 
expressed as C = A * F5(A/B), using the enhancement function F5. 

With the processing using Eq. M3, the dynamic range of the input 
signal IS is not compressed, but it is possible to enhance the local 

20 contrast. Due to the effect of enhancing the local contrast, it is possible 
to give the impression of a "visually" compressed or expanded dynamic 
range. 

It should be noted that <Modified Examples> (ii) «1» to <5» 
discussed above can be similarly adopted in this modified example as 
25 well. That is, in this modified example, the enhancement function F5 is 
a "power function" whose exponent has the same trend as a function 
al(A), a2(A), a3(A), and a4(A,B) discussed in the above <Modified 
Examples> (ii) «1» to <4». As discussed in the above <Modified 
Example> (ii) «5», it is also possible to establish an upper limit or a 
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lower limit for the result of the computation of the enhancement function 
F5. 

(iv) Automatic Setting of Parameters 

In the foregoing embodiment, the target contrast setting portion 
5 604 and the actual contrast setting portion 605 were described as 
allowing the user to set the values of the target contrast Cl and the 
actual contrast C2 through an input interface, for example. It is also 
possible for the target contrast setting portion 604 and the actual 
contrast setting portion 605 to automatically set the values of the target 
10 contrast Cl and the actual contrast C2. 
«1» Display 

An actual contrast setting portion 605 that automatically sets the 
value of the actual contrast C2 will be described below with regard to a 
case in which the display device for displaying the output signal OS is a 

15 display such as a PDP, LCD, or CRT, and the white luminance (white 
level) and black luminance (black level) that can be displayed under 
conditions in which there is no ambient light are known already. 

Fig. 33 shows the actual contrast setting portion 605 that 
automatically sets the value of the actual contrast C2. The actual 

20 contrast setting portion 605 is provided with a luminance measuring 
portion 605a, a memory portion 605b, and a calculation portion 605c. 

The luminance measuring portion 605a is a luminance sensor 
that measures the luminance value of the ambient light in the display 
environment of the display that will display the output signal OS. The 

25 memory portion 605b stores the white luminance (white level) and black 
luminance (black level) that can be displayed under the condition of no 
ambient light by the display for displaying the output signal OS. The 
calculation portion 605c obtains values from the luminance measuring 
portion 605a and the memory portion 605b and uses these to calculate 
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the value of the actual contrast C2. 

An example of the calculation of the calculation portion 605c is 
described below. The calculation portion 605c adds the luminance value 
of the ambient light that it obtains from the luminance measuring 
5 portion 605a to each of the luminance value of the black level and the 
luminance value of the white level stored in the memory portion 605b. 
The calculation portion 605c then divides the result obtained by this 
addition to the luminance value of the white level by the result obtained 
by this addition to the luminance value of the black level, and outputs 
10 the value that is obtained as the value [n] of the actual contrast C2. 
Thus, the value [n] of the actual contrast C2 indicates the contrast value 
that is displayed by the display in a display environment in which there 
is ambient light. 

It is also possible for the memory portion 605b shown in Fig. 33 
15 to store the ratio of the white luminance (white level) and black 
luminance (black level) that can be displayed by the display under the 
condition of no ambient light as the value [m] of the target contrast Cl. 
In this case, the actual contrast setting portion 605 simultaneously also 
functions as the target contrast setting portion 604 for automatically 
20 setting the target contrast Cl. It should be noted that it is also possible 
for the memory portion 605b to not store this ratio and instead for this 
ratio to be calculated by the calculation portion 605c. 
«2» Projector 

An actual contrast setting portion 605 that automatically sets the 
25 value of the actual contrast C2 will be described below with regard to a 
case in which the display device for displaying the output signal OS is a 
projector, for example, and the white luminance (white level) and black 
luminance (black level) that can be displayed under the conditions of no 
ambient light depend on the distance to the screen. 
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Fig. 34 shows an actual contrast setting portion 605 for 
automatically setting the value of the actual contrast C2. The actual 
contrast setting portion 605 is provided with a luminance measuring 
portion 605d and a control portion 605e. 
5 The luminance measuring portion 605d is a luminance sensor 

that measures the luminance value in the display environment of the 
output signal OS that is displayed by the projector. The control portion 
605e causes the projector to perform a display of the white level and the 
black level. It also obtains the luminance values when those levels are 

10 displayed from the luminance measuring portion 605d and calculates the 
value of the actual contrast C2. 

An example of the operation of the control portion 605e is 
described using Fig. 35. First, the control portion 605e operates the 
projector in a display environment in which ambient light is present and 

15 causes it to perform a display of the white level (step S620). The control 
portion 605e obtains the luminance value of the white level that has 
been measured from the luminance measuring portion 605d (step S62l). 
Next, the control portion 605e operates the projector in a display 
environment in which ambient light is present and causes it to perform a 

20 display of the black level (step S622). The control portion 605e obtains 
the luminance value of the black level that has been measured from the 
luminance measuring portion 605d (step S623). The control portion 
605e then calculates the ratio between the luminance value of the white 
level and the luminance value of the black level that have been obtained, 

25 and outputs the result as the value of the actual contrast C2 (step S624). 
The result of this is that the value [n] of the actual contrast C2 indicates 
the contrast value that is displayed by the projector in a display 
environment in which ambient light is present. 

In the same manner as described above, it is also possible to 
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derive the value [m] of the target contrast Cl by calculating the ratio of 
the white level to the black level in a display environment in which no 
ambient light is present. In this case, the actual contrast setting 
portion 605 ends up simultaneously also achieving the function of the 
5 target contrast setting portion 604 for automatically setting the target 
contrast Cl. 

(v) Other Signal Space 

In the foregoing embodiment, the processing in the visual 
processing device 600 was described as being performed with respect to 

10 the luminance of the input signal IS. Here, the present invention is not 
limited to being useful for only a case in which the input signal IS is 
represented by a YCbCr color space. It is also possible for the input 
signal IS to be represented in YUV color space, Lab color space, Luv color 
space, YIQ color space, XYZ color space, or YPbPr color space, for 

15 example. In the case of these, it is possible to execute the processing 
described in the foregoing embodiment with respect to the luminance or 
brightness of that color space. 

Also, if the input signal IS is represented in RGB color space, 
then the processing by the visual processing portion 600 can also be 

20 performed independently for each RGB component. That is, processing 
by the target contrast transformation portion 601 is performed 
independently for the RGB components of the input signal IS, and the 
RGB components of the target contrast signal JS are output. Also, 
processing by the transformed signal processing portion 602 is performed 

25 independently for each of the RGB components of the target contrast 
signal JS, and the RGB components of the visually processed signal KS 
are output. Further, processing by the actual contrast transformation 
portion 603 is performed independently for the RGB components of the 
visually processed signal KS, and the RGB components of the output 
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signal OS are output. Here, the same value for the target contrast Cl 
and for the actual contrast C2 can be used to process each of the RGB 
components. 

(vi) Color Difference Correction 
5 The visual processing device 600 can be further provided with a 

color difference correcting portion in order to prevent the hue of the 
output signal OS from differing from the hue of the input signal IS due 
to the impact of the luminance component that has been processed by the 
transformed signal processing portion 602. 

10 Fig. 36 shows the visual processing device 600 provided with a 

color difference correcting portion 608. It should be noted that the 
structural components that are the same as those of the visual 
processing device 600 shown in Fig. 24 are assigned the same reference 
numerals. The input signal IS shall have a YCbCr color space, in which 

15 the Y component is subjected to the same processing as described in the 
above embodiment. The color difference correcting portion 608 is 
described below. 

The color difference correcting portion 608 receives the target 
contrast signal JS as a first input (value [Yin]), the visually processed 

20 signal KS as a second input (value [Yout]), the Cb component of the input 
signal IS as a third input (value [CBin]), and the Cr component of the 
input signal IS as a fourth input (value [CRin]), and outputs the Cb 
component after color difference correction as a first output (value 
[CBout]), and the Cr component after color difference correction as a 

25 second output (value [CRout]). 

Fig. 37 shows an overview of color difference correction. The 
color difference correcting portion 608 has four inputs [Yin], [Yout], 
[CBin], and [CRin], and by performing computations with these four 
inputs, obtains two outputs [CBout] and [CRout]. 
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[CBout] and [CRout] are derived based on the following equations 
for correcting [CBin] and [CRin] based on the difference and the ratio 
between [Yin] and [Yout]. 

[CBout] is derived based on al * ([Yout] -[Yin]) * [CBin] + a2 * 
5 (l-[Yout]/[Yin]) * [CBin] + a3 * ([Yout] -[Yin]) * [CRin] + a4 * 
(l-[Yout]/[YinD * [CRin] + [CBin] (hereinafter, referred to as the CB 
equation). 

[CRout] is derived based on a5 * ([Yout] -[Yin]) * [CBin] + a6 * 
(l-[Yout]/[Yin]) * [CBin] + a7 * ([Yout] -[Yin]) * [CRin] + a8 * 
10 (l-[Yout]/[YinD * [CRin] + [CRin] (hereinafter, referred to as the CR 
equation). 

The values that are used for the coefficients al to a8 in the CB 
equation and the CR equation are determined by a calculating device 
outside the visual processing device 600, for example, in advance by an 

15 estimation computation that is described below. 

The computation for estimating the coefficients al to a8 with a 
calculating device or the like is described below using Fig. 38. 

First, the four inputs of [Yin], [Yout], [CBin], and [CRin] are 
obtained (step S630). The value of each of these inputs is data that has 

20 been prepared in advance to determine the coefficients al to a8. For 
example, [Yin], [CBin], and [CRin] can each be a value that is obtained 
by decimating all of the obtainable values at a predetermined interval. 
As [Yout], it is possible to use a value that is obtained by decimation at a 
predetermined interval of the values that can be output when the value 

25 of [Yin] has been input to the transformed signal processing portion 602. 
The data that has been prepared in this manner are obtained as the four 
inputs. 

The [Yin], [CBin], and [CRin] that are obtained are transformed 
into a Lab color space and the chromaticity values [Ain] and [Bin] in the 
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transformed Lab color space are calculated (step S63l). 

Next, default coefficients al to a8 are used to perform the 
calculations of the "CB equation" and the "CR equation," from these 
obtaining values for [CBout] and [CRout] (step S632). These obtained 
5 values and [Yout] are transformed into Lab color space, and the 
chromaticity values [Aout] and [Bout] in the transformed Lab color space 
are calculated (step S633). 

Next, the chromaticity values [Ain] , [Bin] , [Aout] , and [Bout] that 
have been calculated are used to perform calculation of an evaluation 

10 function (step S634), and then it is determined whether or not the value 
of the evaluation function is equal to or less than a predetermined 
threshold value. Here, the evaluation function is a function whose 
value is small if the hue change is small between [Ain] and [Bin] and 
between [Aout] and [Bout], and for example it is the squared- sum of the 

15 deviation between each of these components. More specifically, it can be 
([Ain]-[Aout]) A 2 + ([Bin]-[Bout]) A 2. 

If the value of the evaluation function is greater than the 
predetermined threshold value (step S635), then the coefficients al to a8 
are corrected (step S636), and the computations of the steps S632 to 

20 S635 are repeated using the new coefficients. 

If the value of the evaluation function is less than the 
predetermined threshold value (step S635), then the coefficients al to a8 
that have been used for calculating the evaluation function are output as 
the results of the estimation computation (step S637). 

25 It should be noted that in the estimation calculation, any one of 

the combinations of the four inputs [Yin], [Yout], [CBin], and [CRin], 
which are prepared in advance, can be used to estimate the coefficients 
al through a8, but it is also possible to use a plurality of combinations to 
perform the above processing and then output the coefficients al through 
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a8 having the smallest evaluation function as the outcome of the 
estimation calculation. 

[Modified Examples of Color Difference Correction] 

«1» 

5 With the color difference correcting portion 608, [Yin] is taken as 

the value of the target contrast signal JS, [Yout] is taken as the value of 
the visually processed signal KS, [CBin] is taken as the value of the Cb 
component of the input signal IS, [CRin] is taken as the value of the Cr 
component of the input signal IS, [CBout] is taken as the value of the Cb 
10 component of the output signal OS, and [CRout] is taken as the value of 
the Cr component of the output signal OS. Here, it is also possible for 
[Yin], [Yout], [CBin], [CRin], [CBout], and [CRout] to represent other 
signal values. 

For example, if the input signal IS is in RGB color space, then the 
15 target contrast transformation portion 601 (see Fig. 24) performs 
processing with respect to each of these components of the input signal 
IS. In this case, the processed RGB color space signal can be 
transformed into a YCbCr color space signal in which [Yin] is the value 
of the Y component, [CBin] is the value for the Cb component, and [CRin] 
20 is the value of the Cr component. 

Further, if the output signal OS is a signal in RGB color space, 
then derived values for [Yout], [CBout], and [CRout] are transformed into 
an RGB color space, and each of these components is transformed by the 
actual contrast transformation portion 603 to produce the output signal 
25 OS. 

«2» 

It is also possible for the color difference correcting portion 608 to 
use the ratio of the signal values before and after processing by the 
transformed signal processing portion 602 to correct each of the R, G, 
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and B components that are input to the color difference correcting 
portion 608. 

A modified example of the structure of the visual processing 
device 600 is described using Fig. 39. It should be noted that portions 
5 that attain substantially the same function as those of the visual 
processing device 600 shown in Fig. 36 are assigned identical reference 
numerals and shall not be described here. A feature of the 
configuration of the visual processing device 600 of this modified 
example is that it is provided with a luminance signal creation portion 
10 610. 

Each component of the input signal IS, which is a signal in RGB 
color space, is transformed to a target contrast signal JS, which is a 
signal in RGB color space, in the target contrast transformation portion 
601. This process is described in detail above and thus will not be 

15 described here. Here, [Rin], [Gin], and [Bin] serve as the values of the 
respective components of the target contrast signal JS. 

The luminance signal creation portion 610 creates a luminance 
signal having the value [Yin] from the various components of the target 
contrast signal JS. The luminance signal is found by taking the sum of 

20 the values of the R, G, and B components in a particular proportion. 
For example, the value [Yin] can be found by the equation [Yin] = 
0.299*[Rin] + 0.587*[Gin] + 0.114*[Bin]. 

The transformed signal processing portion 602 processes the 
luminance signal having the value [Yin] and outputs the result as a 

25 visually processed signal KS having the value [Yout]. More specifically, 
this processing is the same as the processing by the transformed signal 
processing portion 602 (see Fig. 36), which outputs the visually processed 
signal KS from the target contrast signal JS, and thus will not be 
described here further. 
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The color difference correcting portion 608 uses the luminance 
signal (value [Yin]), the visually processed signal KS (value [Yout]), and 
the target contrast signal JS (value [Rin], [Gin], and [Bin]) to output a 
color difference correction signal (value [Rout], [Gout], [Bout]), which is a 
5 signal in RGB color space. 

More specifically, the color difference correcting portion 608 
calculates the value [Yin] to value [Yout] ratio (the value [[Yout]/[Yin]]). 
The calculated ratio is multiplied to each component of the target 
contrast signal JS (values [Rin], [Gin], and [Bin]) as a color difference 
10 correction coefficient. Accordingly, a color difference correction signal 
(values [Rout], [Gout], and [Bout]) is output. 

The actual contrast transformation portion 603 transforms each 
component of the color difference correction signal, which is a signal in 
RGB color space, transforming it into an output signal OS, which is a 
15 signal in RGB color space. The details for this processing have been 
discussed before, and this will not be described here. 

With the visual processing device 600 of this modified example, 
the transformed signal processing portion 602 performs processing only 
with respect to the luminance signal, and thus it is not necessary to 
20 process each RGB component. Thus, the visual processing burden per 
input signal IS in RGB color space is reduced. 
«3» 

The "CB equation" and "CR equation" are only examples, and 
other equations can also be used. 
25 (vii) Visual Processing Portion 623 

The visual processing portion 623 shown in Fig. 24 can also be 
constituted by a two-dimensional LUT. 

In this case, the two-dimensional LUT stores the values of the 
visually processed signal KS with respect to the values of the target 
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contrast signal JS and the values of the unsharp signal US. More 
specifically, the values of the visual processing signal KS are determined 
based on "Eq. M2," which was described in [First Embodiment] <Profile 
Data> (2) «Second Profile Data». It should be noted that in the "Eq. 
5 M2" the value of the target contrast signal JS serves as the A value and 
the value of the unsharp signal US is used as the value B. 

The visual processing device 600 is provided with a plurality of 
such two-dimensional LUTs in a memory device (not shown). Here, the 
memory device can be internally provided in the visual processing device 

10 600, or it can be connected to an external portion thereof through a wired 
or wireless connection. Each two-dimensional LUT stored on the 
memory device is correlated to the values of the target contrast Cl and 
the values of the actual contrast C2. That is, the same computation as 
that described above in [Second Embodiment] <Transformed Signal 

15 Processing Portion 602> «Action of Transformation Signal Processing 
Portion 602» is performed with respect to each combination of the value 
of the target contrast Cl and the value of the actual contrast C2, and 
stores the result as a two-dimensional LUT. 

When the visual processing portion 623 obtains the values of the 

20 target contrast Cl and the actual contrast C2, the two-dimensional LUT 
that corresponds to those values that have been obtained from among the 
two-dimensional LUTs stored on the memory device is read out. Then, 
the visual processing portion 623 uses the two-dimensional LUT that has 
been read out and performs visual processing. More specifically, the 

25 visual processing portion 623 obtains the value of the target contrast 
signal JS and the value of the unsharp signal US, and reads the value of 
the visually processed signal KS corresponding to those obtained values 
from the two-dimensional LUT and outputs the visually processed signal 
KS. 
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[Third Embodiment] 
<1> 

Applied examples of the visual processing devices, visual 
5 processing methods, and visual processing programs described above in 
the first and second embodiments, and a system using the same, are 
described below as a third embodiment of the present invention. 

The visual processing device is a device for processing images 
that is provided in or connected to a machine that handles images, such 
10 as a computer, a television, a digital camera, a portable telephone, a PDA, 
a printer, or a scanner, and is achieved by an integrated circuit such as 
an LSI. 

More specifically, each functional block of the foregoing 
embodiments can each be separately achieved as single chips, or can be 

15 achieved by a single chip that incorporates some or all of those functions. 
It should be noted that here reference is made to LSIs, but depending on 
the degree of integration, these may also be referred to as ICs, system 
LSIs, super LSIs, or ultra LSIs. 

Further, the manner in which the circuit integration is achieved 

20 is not limited to LSIs, and it is also possible to use a dedicated circuit or 
a general purpose processor. After LSI manufacturing, it is also 
possible to employ a programmable FPGA (Field Programmable Gate 
Array) or a reconfigurable processor in which the connections and 
settings of the circuit cells within the LSI can be reconfigured. 

25 Further, if other technologies that improve upon or are derived 

from semiconductor technology enable integration technology to replace 
LSIs, then naturally it is also possible to integrate the functional blocks 
using that technology. Biotechnology applications are one such 
foreseeable example. 
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The various blocks of the visual processing devices described 
above in the first and second embodiments are for example processed by 
a central processing unit (CPU) provided in the visual processing device. 
Also, a program for performing those processes is stored on a memory 
5 device such as a hard disk or a ROM, and this program is run on the 
ROM, or read to and run on a RAM. 

The two-dimensional LUT 4 in the visual processing device 1 of 
Fig. 1 is stored on a memory device such as a hard disk or ROM, and is 
referenced when necessary. Further, the visual processing device 3 
10 receives profile data that are provided to it from the profile data 
registration device 8, which is connected to the visual processing device 1 
directly or indirectly over a network, and these are registered as the 
two-dimensional LUT 4. 

The visual processing device can also be a device that performs 
15 gradation processing of the images of each frame (each field) that is 
provided in or is connected to a device that handles moving images. 

The visual processing method described in the first embodiment 
is executed on the visual processing device 1. 

The visual processing program is a program for executing visual 
20 processing of images, and is stored on a memory device such as a hard 
disk or ROM in a device provided in or connected to a machine that 
handles images, such as a computer, a television, a digital camera, a 
portable telephone, a PDA, a printer, or a scanner. The program is for 
example provided via a storage medium such as a CD-ROM or via a 
25 network. 

<2> 

It is also possible for the visual processing devices described 
above in the first and second embodiments to be expressed by the 
configurations shown in Fig. 40 and Fig. 41. 
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(1) 

«Configuration» 
Fig. 40 is a block diagram that shows the configuration of a visual 
processing device 910 that achieves the same functions as the visual 
5 processing device 525 shown using Fig. 7 for example. 

In the visual processing device 910, a sensor 911 and a user input 
portion 912 have the same functions as the input device 527 (see Fig. 7). 
More specifically, the sensor 911 is a sensor that detects the ambient 
light in the environment in which the visual processing device 910 has 

10 been placed, or in the environment in which the output signal OS from 
the visual processing device 910 is displayed, and outputs the detected 
value as a parameter PI that expresses the ambient light. The user 
input portion 912 is a device through which the user can set the 
intensity of the ambient light in a stepwise manner, such as "strong, 

15 medium, weak," or in a non-stepwise manner (i.e. in a continuous 
manner), and outputs the value that is set as a parameter PI that 
expresses the ambient light. 

An output portion 914 has the same function as the profile data 
registration portion 526 (see Fig. 7). More specifically, the output 

20 portion 914 is provided with data of a plurality of profiles corresponding 
to the values of the parameter PI, which expresses the ambient light. 
Here, the profile data are data in a table format that give a value of an 
output signal OS with respect to the input signal IS and the signal 
obtained by spatially processing the input signal IS. Further, the 

25 output portion 914 outputs profile data corresponding to the value of the 
parameter PI, which expresses the ambient light, that has been obtained 
to a transformation portion 915 as a luminance adjustment parameter 
P2. 

The transformation portion 915 has the same function as the 
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spatial processing portion 2 and the visual processing portion 3 (see Fig. 
7). The transformation portion 915 receives the luminance of the pixel 
in question (target pixel) to be subjected to visual processing, the 
luminance of the surrounding pixels positioned around the target pixel, 
5 and the luminance adjustment parameter P2 as input, transforms the 
luminance of the target pixel, and outputs the result as the output signal 
OS. 

More specifically, the transformation portion 915 spatially 
processes the target pixel and the surrounding pixels. The 

10 transformation portion 915 also reads the value of the output signal OS 
corresponding to the pixel being processed and the result of the spatial 
processing from the values of the luminance adjustment parameter P2, 
which are in a table format, and outputs this as the output signal OS. 
«Modified Examples» 

15 (1) 

In the above configuration, the luminance adjustment parameter 
P2 is not limited to the above-described profile data. For example, it is 
also possible for the luminance adjustment parameter P2 to be 
coefficient matrix data used when computing the value of the output 

20 signal OS from the luminance of the pixel being processed and the 
luminance of the surrounding pixels. Here, coefficient matrix data are 
data storing the coefficient portion of the function used when computing 
the value of the output signal OS from the luminance of the pixel being 
processed and the luminance of the surrounding pixels. 

25 (2) 

It is not necessary for the output portion 914 to be provided with 
profile data or coefficient matrix data for all the values of the parameter 
PI, which expresses the ambient light. In this case, it is also possible to 
suitably interpolate or extrapolate the profile data e.g. that have been 
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provided in correspondence with the parameter PI, which expresses the 
ambient light, that has been obtained, so as to create appropriate profile 
data e.g. 

(2) 

5 «Configuration» 

Fig. 41 is a block diagram that shows the configuration of a visual 
processing device 920 that has the same functions as the visual 
processing device 600 shown using Fig. 24. 

With the visual processing device 920, the output portion 921 
10 further obtains an outside parameter P3 in addition to the parameter PI, 
which expresses the ambient light, and outputs the luminance 
adjustment parameter P2 based on the parameter PI, which expresses 
the ambient light, and the outside parameter P3. 

Here, the parameter PI that expresses the ambient light is the 
15 same as that described in (l) above. 

The outside parameter P3 is a parameter expressing e.g. the 
visual effect desired by the user who will view the output signal OS. 
More specifically, it is a value such as the contrast desired by the user 
who will view the image (target contrast). Here, the outside parameter 
20 P3 is set by the target contrast setting portion 604 (see Fig. 24). 
Alternatively, it is set using a default value stored on the output portion 
921 in advance. 

The output portion 921 calculates the value of the actual contrast 
through the configuration shown in Fig. 33 and Fig. 34 from the 
25 parameter PI, which expresses the ambient light, and outputs the result 
as the luminance adjustment parameter P2. The output portion 921 
outputs the outside parameter P3 (target contrast) as the luminance 
adjustment parameter P2. The output portion 921 also stores data of a 
plurality of profiles stored in the two-dimensional LUT described in 
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[Second Embodiment] <Modified Example> (vii), and selects profile data 
based on the actual contrast that has been calculated from the outside 
parameter P3 and the parameter PI, which expresses the ambient light, 
and outputs this data, which is in the form of a table, as the luminance 
5 adjustment parameter P2. 

A transformation portion 922 has the same function as the target 
contrast transformation portion 601, the transformed signal processing 
portion 602, and the actual contrast transformation portion 603 (for 
these, see Fig. 24). More specifically, the transformation portion 922 

10 receives the input signal IS (luminance of the pixel being processed and 
luminance of the surrounding pixels) and the luminance adjustment 
parameter P2 as input, and outputs an output signal OS. For example, 
the input signal IS is transformed into the target contrast signal JS (see 
Fig. 24) using the target contrast obtained as the luminance adjustment 

15 parameter P2. It then spatially processes the target contrast signal JS, 
thus deriving the unsharp signal US (see Fig. 24). 

The transformation portion 922 is provided with the visual 
processing portion 623 of the modified example described in [Second 
Embodiment] <Modified Example> (vii), and outputs the visually 

20 processed signal KS (see Fig. 24) from the profile data obtained as the 
luminance adjustment parameter P2, the target contrast signal JS, and 
the unsharp signal US. The visually processed signal KS is then 
transformed into the output signal OS using the actual contrast that is 
obtained as the luminance adjustment parameter P2. 

25 With the visual processing device 920, it is possible to select the 

profile data to be used for visual processing based on the outside 
parameter P3 and the parameter PI, which expresses the ambient light, 
and also, it is possible to compensate for the effect of the ambient light to 
improve the local contrast, even in an environment where ambient light 
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is present, bringing the contrast closer to the contrast desired by the 
user who will view the output signal OS. 
«Modified Examples» 
It should be noted that it is possible to perform the same 
5 modifications as those discussed in (l) with this configuration as well. 

Where necessary, it is also possible to switch the configuration 
described in (l) and the configuration described in (2). This switching 
can be performed using a switch signal from the outside. It is also 
possible to determine which of the configurations to use based on 
10 whether or not the outside parameter P3 is present. 

Although the actual contrast was described as being calculated by 
the output portion 921, it is also possible to adopt a configuration in 
which the value of the actual contrast is directly input to the output 
portion 921. 
15 (3) 

With the configuration shown in Fig. 41, it is further possible to 
adopt means for keeping the input from the output portion 921 to the 
transformation portion 922 from changing abruptly. 

A visual processing device 920' shown in Fig. 42 differs from the 
20 visual processing device 920 shown in Fig. 41 in that it is further 
provided with an adjustment portion 925 that eases the change over time 
in the parameter PI, which expresses the ambient light. The 
adjustment portion 925 receives the parameter PI expressing the 
ambient light as input and outputs a post-adjustment output P4. 
25 By doing this, it becomes possible for the output portion 921 to 

obtain a parameter PI that expresses the ambient light that does not 
change abruptly, and as a result, the change in the output of the output 
portion 921 over time also becomes gentle. 

The adjustment portion 925 is achieved by an IIR filter, for 
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example. Here, with an IIR filter, the value [P4] of the output P4 of the 
adjustment portion 925 is calculated by [P4] = kl*[P4]' + k2*[Pl]. In 
the equation, kl and k2 are each parameters that take a positive value, 
[Pi] is the value of the parameter PI, which expresses the ambient light, 
5 and [P4]' is the value of the delay output (for example, the previous 
output) of the output P4 of the adjustment portion 925. It should be 
noted that the processing in the adjustment portion 925 can also be 
performed using a component other than an IIR filter. 

Further, it is also possible for the adjustment portion 925 to be 

10 means that directly smoothes the change in the luminance adjustment 
parameter P2 over time that is provided on the output side of the output 
portion 921, as in the visual processing device 920" shown in Fig. 43. 

Here, the operation of the adjustment portion 925 is the same as 
that described above. More specifically, the value [P4] of the output P4 

15 of the adjustment portion 925 is calculated by [P4] = k3*[P4]' + k4*[P2]. 
In the equation, k3 and k4 are each parameters that take a positive 
value, [P2] is the value of the luminance adjustment parameter P2, and 
[P4]' is the value of the delay output (for example, the previous output) of 
the output P4 of the adjustment portion 925. It should be noted that 

20 the processing in the adjustment portion 925 can also be performed using 
a component other than an IIR filter. 

With the configuration shown in Figs. 42 and 43, for example, it 
is possible to control the change over time of the parameter Pi 
expressing the ambient light or the luminance adjustment parameter P2. 

25 Thus, it is possible to inhibit sudden parameter fluctuations, even if the 
sensor 911 for detecting the ambient light responds to a person who has 
moved in front of the sensor and for a brief moment the parameters 
significantly change. As a result, flickering in the display screen can be 
inhibited. 
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[Fourth Embodiment] 

An example of the application of the visual processing devices, 
the visual processing methods and the visual processing programs, as 
5 well as a system using the same according to a fourth example of the 
present invention are described with reference to Figs. 44 to 47. 

Fig. 44 is a block diagram showing an overall structure of a 
content providing system exlOO that realizes a content delivering service. 
An area where a communication service is provided is divided into cells 
10 of a desired size, and base stations exl07-exll0 that are fixed radio 
stations are provided in the cells. 

This content providing system exlOO includes a computer exlll, a 
personal digital assistant (PDA) exll2, a camera exll3, a cellular phone 
exll4, a cellular phone with camera exll5 and other equipment that are 
15 connected to the Internet exlOl for example via an internet service 
provider exl02, a telephone network exl04 and base stations 
exl07-exll0. 

However, the content providing system exlOO can adopt any 
combination for connection without being limited to the combination 

20 shown in Fig. 44. In addition, each of the devices can be connected 
directly to the telephone network exl04 without the base stations 
exl07-exll0 that are fixed radio stations. 

The camera exll3 is a device such as a digital video camera that 
can obtain a moving image. In addition, the cellular phone may be any 

25 type of PDC (Personal Digital Communications) method, CDMA (Code 
Division Multiple Access) method, W-CDMA (Wideband- Code Division 
Multiple Access) method, or GSM (Global System for Mobile 
Communications) method, or a cellular phone of PHS (Personal 
Handyphone System). 
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In addition, the streaming server exl03 is connected to the 
camera exll3 via the base station exl09 and the telephone network 
exl04, so that live delivery can be performed on the basis of coded data 
transmitted by a user of the camera exll3. The coding process of the 
5 obtained data may be performed by the camera exll3 or by a server for 
transmitting data. In addition, the moving image data obtained by the 
camera exll6 may be transmitted to the streaming server exl03 via the 
computer exlll. The camera exll6 is a device that can take a still 
image like a digital camera and a moving image. In this case, coding of 

10 the moving image data may be performed by the camera exll6 or by the 
computer exlll. In addition, the coding process may be performed by 
an LSI exll7 in the computer exlll or the camera exll6. Note that it is 
possible to incorporate software for coding and decoding images into a 
storage medium (a CD-ROM, a flexible disk, a hard disk or the like) that 

15 is a recording medium readable by the computer exlll. Furthermore, 
the cellular phone with camera exll5 may transmit the moving image 
data. In this case, the moving image data is coded by the LSI in the 
cellular phone exll5. 

In this content providing system exlOO, content (for example, a 

20 moving image of a music concert) that the user is recording with the 
camera exll3 or the camera exll6 are coded as shown in the 
above-described embodiments and transmitted to the streaming server 
exl03, while the streaming server exl03 delivers a stream of the content 
data to a client who made a request. The client may be the computer 

25 exlll, the PDA exll2, the camera exll3, the cellular phone exll4 or the 
like that can decode the coded data. Thus, in the content providing 
system exlOO, the client can receive and reproduce the coded data. The 
system can realize personal broadcasting when the client receives, 
decodes and reproduces the stream in real time. 
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To display the content, the visual processing devices, the visual 
processing methods and the visual processing programs may be used. 
For example, the computer exlll, the PDA exll2, the camera exll3, the 
cellular phone exll4 or the like may be provided with the visual 
5 processing devices and execute the visual processing methods and the 
visual processing programs. 

The streaming server exl03 may be a device which provides the 
visual processing device with the profile data through the Internet exlOl. 
Moreover, a plurality of the streaming servers exl03 may be exist and 

10 each of the streaming servers exl03 may provide different profile data. 
Furthermore, the streaming server exl03 may produce the profile data. 
If the visual processing device can obtain the profile data through the 
Internet exlOl, it is not necessary for the visual processing device to 
store a large number of profiles for the visual processing in advance, and 

15 this allows the storage capacity for the profiles to be reduced. Further, 
by obtaining the profiles from different servers which locate on the 
Internet exlOl, it is possible for the visual processing device to execute 
different image processing. 

An example regarding a cellular phone will now be described. 

20 Fig. 45 shows the cellular phone exll5 that utilizes the visual 

processing devices of the present invention. The cellular phone exll5 
includes an antenna ex201 for transmitting and receiving radio waves 
with the base station exllO, a camera portion ex203 such as a CCD 
camera that can take a still image, a display portion ex202 such as a 

25 liquid crystal display for displaying images obtained by the camera 
portion ex203 or images received by the antenna ex201 after the image 
data are decoded, a main body portion including a group of operating 
keys ex204, a sound output portion ex208 such as a speaker for 
producing sounds, a sound input portion ex205 such as a microphone for 
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receiving sounds, a recording medium ex207 for storing coded data or 
decoded data such as data of taken moving images or still images, data 
of received e-mails, moving images or still images, and a slot portion 
ex206 that enables the recording medium ex207 to be attached to the 
5 cellular phone exll5. The recording medium ex207 such as an SD card 
includes a plastic case housing a flash memory element that is one type 
of EEPROM (Electrically Erasable and Programmable Read Only 
Memory) nonvolatile memory that is electronically rewritable and 
erasable. 

10 Furthermore, the cellular phone exll5 will be described with 

reference to Fig. 46. The cellular phone exll5 includes a main 
controller portion ex311 for controlling each portion of the main body 
portion having the display portion ex202 and the operating keys ex204, a 
power source circuit portion ex310, an operational input controller 

15 portion ex304, an image coding portion ex312, a camera interface portion 
ex303, an LCD (Liquid Crystal Display) controller portion ex302, an 
image decoding portion ex309, a multiplex separation portion ex308, a 
recording and reproduction portion ex307, a modem circuit portion ex306 
and a sound processing portion ex305, which are connected to each other 

20 via a synchronizing bus ex313. 

When the user turns on a clear and power key, the power source 
circuit portion ex310 supplies power from a battery pack to each portion 
so that the digital cellular phone with camera exll5 is activated. 

The cellular phone exll5 converts a sound signal collected by the 

25 sound input portion ex205 during a sound communication mode into 
digital sound data by the sound processing portion ex305 under control 
of the main controller portion ex311 that includes a CPU, a ROM and a 
RAM. The digital sound data are processed by the modem circuit 
portion ex306 as a spectrum spreading process and are processed by the 
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transmission and reception circuit portion ex301 as a digital to analog 
conversion process and a frequency conversion process. After that, the 
data are transmitted via the antenna ex201. In addition, the cellular 
phone exll5 amplifies a signal that is received by the antenna ex201 
5 during the sound communication mode and performs the frequency 
conversion process and an analog to digital conversion process on the 
data, which is processed by the modem circuit portion ex306 as a 
spectrum inverse spreading process and is converted into a analog sound 
signal by the sound processing portion ex305. After that, the analog 

10 sound signal is delivered by the sound output portion ex208. 

Furthermore, when transmitting electronic mail during a data 
communication mode, text data of the electronic mail are entered by 
using the operating keys ex204 of the main body portion and are given to 
the main controller portion ex311 via the operational input controller 

15 portion ex304. The main controller portion ex311 performs the 
spectrum spreading process on the text data by the modem circuit 
portion ex306 and performs the digital to analog conversion process and 
the frequency conversion process by the transmission and reception 
circuit portion ex301. After that, the data are transmitted to the base 

20 station exllO via the antenna ex201. 

When transmitting image data during the data communication 
mode, the image data obtained by the camera portion ex203 are supplied 
to the image coding portion ex312 via the camera interface portion ex303. 
In addition, if the image data are not transmitted, it is possible to 

25 display the image data obtained by the camera portion ex203 directly by 
the display portion ex202 via the camera interface portion ex303 and an 
LCD controller portion ex302. 

The image coding portion ex312 converts the image data supplied 
from the camera portion ex203 into the coded image data by compressing 
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and coding the data, and the coded image data are supplied to the 
multiplex separation portion ex308. In addition, the cellular phone 
exll5 collects sounds by the sound input portion ex205 while the camera 
portion ex203 is taking the image, and the digital sound data is supplied 
5 from the sound processing portion ex305 to the multiplex separation 
portion ex308. 

The multiplex separation portion ex308 performs multiplexing of 
the coded image data supplied from the image coding portion ex312 and 
the sound data supplied from the sound processing portion ex305 by a 

10 predetermined method. Multiplexed data obtained as a result are 
processed by the modem circuit portion ex306 as a spectrum spreading 
process and are processed by the transmission and reception circuit 
portion ex301 as a digital to analog conversion process and a frequency 
conversion process. After that, the data are transmitted via the 

15 antenna ex201. 

When receiving moving image file data that are linked to a web 
page during the data communication mode, a signal received from the 
base station exllO via the antenna ex201 is processed by the modem 
circuit portion ex306 as a spectrum inverse spreading process. 

20 Multiplexed data obtained as a result are supplied to the multiplex 
separation portion ex308. 

In addition, in order to decode multiplexed data received via the 
antenna ex201, the multiplex separation portion ex308 separates a coded 
bit stream of image data in the multiplexed data from a coded bit stream 

25 of sound data. Then, the multiplex separation portion ex308 supplies 
the coded image data to the image decoding portion ex309 via the 
synchronizing bus ex313 and supplies the sound data to the sound 
processing portion ex305. 

Next, the image decoding portion ex309 generates reproduction 
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moving image data by decoding the coded bit stream of the image data 
and supplies the data to the display portion ex202 via the LCD controller 
portion ex302. Thus, the moving image data included in a moving 
image file that is linked to a home page can be displayed. In this case, 
5 the sound processing portion ex305 converts the sound data into an 
analog sound signal, which is supplied to the sound output portion 
ex208. Thus, sound data included in the moving image file that is 
linked to a home page can be reproduced. 

Note that the image decoding portion ex309 may be provided with 

10 the visual processing device shown in the above-described embodiments. 

Note that the present invention is not limited to the example of 
the system described above. Digital broadcasting by satellite or 
terrestrial signals has been a recent topic of discussion. As shown in 
Fig. 47, the visual processing devices, the visual processing methods and 

15 the visual processing programs of the present invention can be 
incorporated into the digital broadcasting system, too. 

More specifically, in a broadcast station ex409, a coded bit stream 
of image information is sent to a communication or a broadcasting 
satellite ex410 via a radio wave. The broadcasting satellite ex410 that 

20 received the coded bit stream of image information sends radio waves for 
broadcasting. These radio waves are received by an antenna ex406 of a 
house equipped with a satellite broadcasting reception facility, and a 
device such as a television set (a receiver) ex401 or a set top box (STB) 
ex407 decodes the coded bit stream and reproduces the same. In 

25 addition, the television set (the receiver) ex401 or the set top box (STB) 
ex407 may be provided with the visual processing devices shown in the 
above-described embodiments, use the visual processing methods shown 
in the above-described embodiments or execute the visual processing 
programs shown in the above-described embodiments. In addition, a 
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reproduction device ex403 for reading and decoding a coded bit stream 
that is recorded on a storage medium ex402 such as a CD or a DVD that 
is a recording medium may be equipped with the visual processing 
devices, the visual processing methods or the visual processing programs 
5 shown in the above-described embodiments. In this case, the 
reproduced image signal and text track are displayed on a monitor ex404. 
In addition, it is possible to mount the visual processing devices, the 
visual processing methods or the visual processing programs shown in 
the above-described embodiments, in a set top box ex407 that is 

10 connected to a cable ex405 for a cable television or the antenna ex406 for 
a satellite or surface wave broadcasting, so that the image can be 
reproduced on a monitor ex408 of the television set. In this case, it is 
possible to incorporate the visual processing devices shown in the 
above-described embodiments not into the set top box but into the 

15 television set. In addition, it is possible that a car ex412 equipped with 
an antenna ex411 receives a signal from the broadcasting satellite ex410 
or the base station exl07 and reproduces the moving image on a display 
of a navigation system ex413 in the car ex412. 

Furthermore, it is possible to encode the image signal and record 

20 the encoded image signal in a recording medium. As a specific example, 
there is a recorder ex420 such as a DVD recorder for recording image 
signals on a DVD disk ex421 or a disk recorder for recording image 
signals on a hard disk. Furthermore, it is possible to record on an SD 
card ex422. In addition, in case that the recorder ex420 includes the 

25 decoding apparatus of the present invention, it is possible to reproduce 
image signals recorded on a DVD disk ex421 or a SD card ex422 via the 
image signal processing device, so as to display on the monitor ex408. 

Note that in the structure of the navigation system ex413 shown 
in Fig. 46, the camera portion ex203, the camera interface portion ex303 
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and the image coding portion ex312 can be omitted. This can be also 
applied to the computer exlll and the television set (the receiver) ex401. 

In addition, the terminal device such as the cellular phone exll4 
may include three types of assemblies. A first type is a transmission 
5 and reception terminal having both the coder and the decoder, a second 
type is a transmission terminal having only a coder and a third type is a 
reception terminal having only a decoder. 

Thus, the visual processing devices, the visual processing 
methods or the visual processing programs shown in the above-described 
10 embodiments can be used for any device and system described above, so 
that effects described above can be obtained. 



[Attachment] 

The present invention can also be expressed as follows. 
<Content of the Attachment> 

(Attachment l) 
A visual processing device comprising: 

image signal processing means that performs spatial processing 
of an image signal that has been received as input, and outputs a 
processed signal; and 

signal computation means that outputs an output signal based on 
a computation for emphasizing the respective differences in the image 
signal and the processed signal after each has been transformed by a 
predetermined transformation. 

(Attachment 2) 
The visual processing device according to attachment 1, 
wherein the signal computation means computes a value C of the 
output signal with respect to a value A of the image signal, a value B of 
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the processed signal, a transformation function Fl, an inverse 
transformation function F2 of the transformation function Fl, and the 
enhancement function F3, according to the equation F2(Fl(A) + 
F3(F1(A)-F1(B))). 

5 

(Attachment 3) 
The visual processing device according to attachment 2, 
wherein the transformation function Fl is a logarithmic function. 

10 (Attachment 4) 

The visual processing device according to attachment 2, 
wherein the inverse transformation function F2 is a gamma 
correction function. 



15 (Attachment 5) 

The visual processing device according to any one of attachments 

2 to 4, 

wherein the signal computation means has signal space 
transformation means that transforms the signal space of the image 

20 signal and the processed signal, enhancement processing means that 
performs enhancement processing of a difference signal of the difference 
between the image signal after transformation and the processed signal 
after transformation, and inverse transformation means that performs 
inverse transformation of the signal space with respect to a sum signal 

25 obtained by summing the image signal after transformation and the 
difference signal after enhancement processing and outputs the result as 
the output signal. 



(Attachment 6) 
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A visual processing device comprising: 

image signal processing means that performs spatial processing 
of an image signal that has been received as input, and outputs a 
processed signal; and 
5 signal computation means that outputs an output signal based on 

a computation for emphasizing the ratio between the image signal and 
the processed signal. 

(Attachment 7) 

10 The visual processing device according to attachment 6, 

wherein the signal computation means outputs the output signal 
based on said calculation in which dynamic range compression of the 
image signal is further performed. 

15 (Attachment 8) 

The visual processing device according to attachment 6 or 7, 
wherein the signal computation means computes a value C of the 
output signal with respect to a value A of the image signal, a value B of 
the processed signal, a dynamic range compression function F4, and the 
20 enhancement function F5, according to the equation F4(A) * F5(A/B). 

(Attachment 9) 
The visual processing device according to attachment 8, 
wherein the dynamic range compression function F4 is a direct 
25 proportion function of a proportion coefficient 1. 

(Attachment 10) 
The visual processing device according to attachment 8, 
wherein the dynamic range compression function F4 is a 
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monotonically increasing function. 



(Attachment 11) 
The visual processing device according to attachment 10, 
wherein the dynamic range compression function F4 is a concave 
function. 

(Attachment 12) 
The visual processing device according to attachment 8, 
wherein the dynamic range compression function F4 is a power 
function. 

(Attachment 13) 
The visual processing device according to attachment 12, 
wherein the exponent of the power function in the dynamic range 
compression function F4 is determined based on a target contrast value, 
which is a target value of the contrast when displaying the image, and 
an actual contrast value, which is a contrast value in the display 
environment when displaying the image. 

(Attachment 14) 

The visual processing device according to any one of attachments 

8 to 13, 

wherein the enhancement function F5 is a power function. 

(Attachment 15) 
The visual processing device according to attachment 14, 
wherein the exponent of the power function in the enhancement 
function F5 is determined based on a target contrast value, which is a 
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target value of the contrast when displaying the image, and an actual 
contrast value, which is a contrast value in the display environment 
when displaying the image. 

5 (Attachment 16) 

The visual processing device according to attachment 14 or 15, 
wherein the exponent of the power function in the enhancement 
function F5 is a value that decreases monotonically with respect to the 
value A of the image signal in a case where the value A of the image 
10 signal is larger than the value B of the processed signal. 

(Attachment 17) 
The visual processing device according to attachment 14 or 15, 
wherein the exponent of the power function in the enhancement 
15 function F5 is a value that increases monotonically with respect to the 
value A of the image signal in a case where the value A of the image 
signal is smaller than the value B of the processed signal. 

(Attachment 18) 

20 The visual processing device according to attachment 14 or 15, 

wherein the exponent of the power function in the enhancement 
function F5 is a value that increases monotonically with respect to the 
value A of the image signal in a case where the value A of the image 
signal is larger than the value B of the processed signal. 

25 

(Attachment 19) 
The visual processing device according to attachment 14 or 15, 
wherein the exponent of the power function in the enhancement 
function F5 is a value that increases monotonically with respect to the 
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absolute value of the difference between the value A of the image signal 
and the value B of the processed signal. 

(Attachment 20) 

The visual processing device according to any one of attachment 
14 to 19, 

wherein at least one of a maximum value or a minimum value of 
the enhancement function F5 is kept within a predetermined range. 

(Attachment 21) 
The visual processing device according to attachment 8, 
wherein the signal computation means has enhancement 
processing means that performs enhancement processing of a divided 
signal that is obtained by dividing the image signal by the processed 
signal, and output processing means that outputs the output signal 
based on the image signal and the divided signal that has been subjected 
to enhancement processing. 

(Attachment 22) 
The visual processing device according to attachment 21, 
wherein the output processing means obtains the product of the 

image signal and the divided signal that has been subjected to 

enhancement processing. 

(Attachment 23) 
The visual processing device according to attachment 21, 
wherein the output processing means includes DR compression 
means that performs dynamic range (DR) compression with respect to 
the image signal, and obtains the product of the image signal after DR 
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compression and the divided signal that has been subjected to 
enhancement processing. 

(Attachment 24) 

The visual processing device according to any one of attachments 
8 to 23, further comprising: 

first transformation means that transforms input image data of a 
first predetermined range to a second predetermined range, and outputs 
this as the image signal; and 

second transformation means that transforms the output signal 
of a third predetermined range to a fourth predetermined range, and 
outputs this as output image data; 

wherein the second predetermined range is determined based on 
the target contrast value, which is the target value of the contrast when 
the image is displayed; and 

wherein the third predetermined range is determined based on 
the actual contrast value, which is the contrast value in the display 
environment when the image is displayed. 

(Attachment 25) 
The visual processing device according to attachment 24, 
wherein the dynamic range compression function F4 is a function 

that transforms image signal of the second predetermined range into the 

output signal of the third predetermined range. 

(Attachment 26) 
The visual processing device according to attachment 24 or 25, 
wherein the first transformation means transforms the minimum 
value and the maximum value of the first predetermined range into the 
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minimum value and the maximum value, respectively, of the second 
predetermined range! and 

wherein the second transformation means transforms the 
minimum value and the maximum value of the third predetermined 
5 range into the minimum value and the maximum value, respectively, of 
the fourth predetermined range. 

(Attachment 27) 
The visual processing device according to attachment 26, 
10 wherein the transformations by the first transformation means 

and the second transformation means are linear transformations. 

(Attachment 28) 

The visual processing device according to any one of attachments 
15 24 to 27, further comprising: 

setting means that sets the third predetermined range. 

(Attachment 29) 
The visual processing device according to attachment 28, 
20 wherein the setting means includes storage means that stores a 

dynamic range of the display device for displaying an image, and 
measuring means that measures the luminance of the ambient light in 
the display environment when displaying an image. 

25 (Attachment 30) 

The visual processing device according to attachment 28, 
wherein the setting means includes measuring means that 
measures the luminance in the display environment of the display device 
for displaying an image when displaying black level and when displaying 
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white level. 



(Attachment 31) 
A visual processing device comprising: 
5 input signal processing means that performs spatial processing of 

an image signal that has been received as input, and outputs a processed 
signal; and 

signal computation means that outputs an output signal based on 
a computation for enhancing a difference between the image signal and 
10 the processed signal in accordance with a value of the image signal. 



(Attachment 32) 
The visual processing device according to attachment 31, 
wherein the signal computation means outputs the output signal 
15 based on a computation that adds a value that is obtained by 
compressing the dynamic range of the image signal to the enhanced 
value from the computation for enhancing. 



(Attachment 33) 

20 The visual processing device according to attachment 31 or 32, 

wherein the signal computation means computes a value C of the 
output signal with respect to a value A of the input signal, a value B of 
the processed signal, an enhancement amount adjustment function F6, 
an enhancement function F7, and a dynamic range compression function 

25 F8, according to the equation F8(A) + F6(A) * F7(A-B). 

(Attachment 34) 
The visual processing device according to attachment 33, 
wherein the dynamic range compression function F8 is a direct 



129 



proportion function of a proportion coefficient 1. 

(Attachment 35) 
The visual processing device according to attachment 33, 
wherein the dynamic range compression function F8 is a 
monotonically increasing function. 

(Attachment 36) 
The visual processing device according to attachment 35, 
wherein the dynamic range compression function F8 is a concave 
power function. 

(Attachment 37) 
The visual processing device according to attachment 33, 
wherein the dynamic range compression function F8 is a power 
function. 

(Attachment 38) 
The visual processing device according to attachment 33, 
wherein the signal computation means has enhancement 
processing means that performs enhancement processing of a difference 
signal of the difference between the image signal and the processed 
signal in accordance with the pixel value of the image signal, and output 
processing means that outputs the output signal based on the image 
signal and the enhanced difference signal. 

(Attachment 39) 
The visual processing device according to attachment 38, 
wherein the output processing means performs a summing 
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process of taking the sum of the image signal and the enhanced 
difference signal. 

(Attachment 40) 

5 The visual processing device according to attachment 38, 

wherein the output processing means includes DR compression 
means that performs dynamic range (DR) compression with respect to 
the image signal, and performs a summing process of taking the sum of 
the DR compressed image signal and the enhanced difference signal. 

10 

(Attachment 41) 
A visual processing device comprising: 

input signal processing means that performs spatial processing of 
an image signal that has been received as input, and outputs a processed 
15 signal; and 

signal computation means that outputs an output signal based on 
a computation that adds a value obtained by correcting the gradation of 
the image signal to the value obtained by enhancing the difference 
between the image signal and the processed signal. 

20 

(Attachment 42) 
The visual processing device according to attachment 41, 
wherein the signal computation means computes a value C of the 
output signal with respect to a value A of the input signal, a value B of 
25 the processed signal, an enhancement function Fll, and a gradation 
correction function F12, according to the equation F12(A) + Fll(A-B). 

(Attachment 43) 
The visual processing device according to attachment 42, 
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wherein the signal computation means has enhancement 
processing means that performs enhancement processing of a difference 
signal of the difference between the image signal and the processed 
signal, and summing means that takes the sum of the 
5 gradation-corrected image signal and the enhanced difference signal and 
outputs the result as an output signal. 

(Attachment 44) 
A visual processing method comprising: 
10 a first transformation step of transforming an input image data 

of a first predetermined range to a second predetermined range, and 
taking this as an image signal; 

a signal computation step of outputting an output signal of a 
third predetermined range based on a computation that includes at least 
15 one of a computation for compressing the dynamic range of the image 
signal, and a computation for enhancing a ratio between the image 
signal and a processed signal that is obtained by spatially processing the 
image signal; and 

a second transformation step of transforming the output signal of 
20 the third predetermined range to a fourth predetermined range, and 
taking this as output image data; 

wherein the second predetermined range is determined based on 
a target contrast value, which is a target value of the contrast when the 
image is displayed; and 
25 wherein the third predetermined range is determined based on an 

actual contrast value, which is a contrast value in the display 
environment when the image is displayed. 

(Attachment 45) 
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A visual processing device comprising: 

first transformation means that transforms input image data of a 
first predetermined range to a second predetermined range, and takes 
this as an image signal; 
5 signal computation means that outputs an output signal of a 

third predetermined range based on a computation that includes at least 
one of a computation for compressing the dynamic range of the image 
signal, and a computation for enhancing a ratio between the image 
signal and a processed signal that is obtained by spatially processing the 
10 image signal; and 

second transformation means that transforms the output signal 
of the third predetermined range to a fourth predetermined range, and 
takes this as output image data! 

wherein the second predetermined range is determined based on 
15 a target contrast value, which is a target value of the contrast when the 
image is displayed; and 

wherein the third predetermined range is determined based on an 
actual contrast value, which is a contrast value in the display 
environment when the image is displayed. 

20 

(Attachment 46) 

A visual processing program for causing a computer to perform 
visual processing, the visual processing program causing the computer to 
perform a visual processing method comprising: 
25 a first transformation step of transforming an input image data 

of a first predetermined range to a second predetermined range, and 
taking this as an image signal; 

a signal computation step of outputting an output signal of a 
third predetermined range based on a computation that includes at least 
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one of a computation for compressing the dynamic range of the image 
signal, and a computation for enhancing a ratio between the image 
signal and a processed signal that is obtained by spatially processing the 
image signal; and 

5 a second transformation step of transforming the output signal of 

the third predetermined range to a fourth predetermined range, and 
taking this as output image data; 

wherein the second predetermined range is determined based on 
a target contrast value, which is a target value of the contrast when the 
10 image is displayed; and 

wherein the third predetermined range is determined based on an 
actual contrast value, which is a contrast value in the display 
environment when the image is displayed. 



15 <Description of the Attachment> 

The visual processing device according to attachment 1 is 
provided with image signal processing means and signal computation 
means. The image signal processing means performs spatial processing 
of an image signal that has been received as input, and outputs a 

20 processed signal. The signal computation means outputs an output 
signal based on a computation for emphasizing the respective differences 
in the image signal and the processed signal after each has been 
transformed by a predetermined transformation. 

Here, spatial processing for example refers to processing in which 

25 a low pass spatial filter is adopted for the image signal that is received 
as input, or processing for deriving a mean value, a maximum value, or a 
minimum value of the pixel being processed and the surrounding pixels 
of the image signal that is received as input (the same applies in the 
following). Also, the computation for enhancing is for example a 
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computation for adjusting the gain, a computation for inhibiting extreme 
contrasts, or a computation for suppressing small amplitude noise 
components (the same applies in the following). 

With the visual processing device of this invention, it is possible 
5 to enhance the respective differences in the image signal and the 
processed signal after these have been transformed to a separate space. 
Thus, it is possible to achieve enhancement that is suited for the viewing 
properties, for example. 

The visual processing device according to attachment 2 is the 

10 visual processing device according to attachment 1 in which the signal 
computation means computes a value C of the output signal with respect 
to a value A of the image signal, a value B of the processed signal, a 
transformation function Fl, an inverse transformation function F2 of the 
transformation function Fl, and the enhancement function F3, according 

15 to the equation F2(Fl(A) + F3(Fl(A)-Fl(B))). 

The enhancement function F3 is for example a function for 
adjusting the gain, a function for inhibiting extreme contrasts, or a 
function for suppressing small amplitude noise components. 

Here, the value C of the output signal indicates the following. 

20 That is, the value A of the image signal and the value B of the processed 
signal are transformed into values in different spaces by the 
transformation function Fl. The difference between the value of the 
image signal and the value of the processed signal after transformation 
represents a sharp signal, for example, in a separate space. The 

25 difference between the image signal and the processed signal after 
transformation, in which they are enhanced by the enhancement 
function F3, is added to the transformed image signal. Thus, the value 
C of the output signal indicates a value that is obtained by enhancing 
the sharp signal component in a separate space. 
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With the visual processing device of this invention, it is for 
example possible to use the value A of the image signal and the value B 
of the processed signal, which have been transformed to separate spaces, 
to perform processing such as edge enhancement or contrast 
5 enhancement in a separate space. 

The visual processing device according to attachment 3 is the 
visual processing device according to attachment 2 in which the 
transformation function Fl is a logarithmic function. 

Here, human visual performance in general is logarithmic. Thus, 
10 by processing the image signal and the processed signal by transforming 
to logarithmic space, it is possible to perform visual processing that is 
suited for our visual characteristics. 

With the visual processing device of this invention, it is possible 
to perform contrast enhancement having a high visual effect or dynamic 
15 range compression that maintains local contrast. 

The visual processing device according to attachment 4 is the 
visual processing device according to attachment 2 in which the inverse 
transformation function F2 is a gamma correction function. 

Image signals in general are subjected to gamma correction 
20 through a gamma correction function based on to the gamma 
characteristics of the device that inputs and outputs the image signal. 

With the visual processing device of this invention, with the 
transformation function Fl it is possible to exclude gamma correction of 
the image signal and perform processing using linear characteristics. 
25 Thus, it is possible to correct optical blurring. 

The visual processing device according to attachment 5 is the 
visual processing device according to any one of attachments 2 to 4, in 
which the signal computation means has signal space transformation 
means, enhancement processing means, and inverse transformation 
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means. The signal space transformation means transforms the signal 
space of the image signal and the processed signal. The enhancement 
processing means performs enhancement processing of a differential 
signal between the image signal after transformation and the processed 
5 signal after transformation. The inverse transformation means 
performs inverse transformation of the signal space with respect to a 
sum signal obtained by summing the image signal after transformation 
and the differential signal after enhancement processing and outputs the 
result as the output signal. 

10 With the visual processing device of this invention, the signal 

space transformation means transforms the signal space of the image 
signal and the processed signal using the transformation function Fl. 
The enhancement processing means performs enhancement processing of 
a differential signal between the image signal after transformation and 

15 the processed signal after transformation. The inverse transformation 
means performs inverse transformation of the signal space with respect 
to a sum signal obtained by summing the image signal after 
transformation and the difference signal after enhancement processing, 
using the inverse transformation function F2. 

20 The visual processing device according to attachment 6 comprises 

input signal processing means and signal computation means. The 
input signal processing means performs spatial processing of an image 
signal that has been received as input, and outputs a processed signal. 
The signal computation means outputs an output signal based on a 

25 computation for emphasizing the ratio between the image signal and the 
processed signal. 

With the visual processing device of this invention, the ratio of 
the image signal to the processed signal expresses the sharp component 
of the image signal. Thus, it is for example possible to perform visual 
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processing for enhancing the sharp component. 

The visual processing device according to attachment 7 is the 
visual processing device according to attachment 6 in which the signal 
computation means outputs the output signal based on the calculation in 
5 which dynamic range compression of the image signal is further 
performed. 

With the visual processing device of this invention, it is possible 
to enhance the sharp component of the image signal, which expresses the 
ratio of the image signal to the processed signal, while performing 

10 compression of the dynamic range. 

The visual processing device according to attachment 8 is the 
visual processing device according to attachment 6 or 7, in which the 
signal computation means computes a value C of the output signal with 
respect to a value A of the image signal, a value B of the processed signal, 

15 a dynamic range compression function F4, and the enhancement function 
F5, according to the equation F4(A) * F5(A/B). 

Here, the value C of the output signal shows the following. That 
is, the amount (A/B) obtained by dividing the value A of the image signal 
by the value B of the processed signal expresses the sharp signal, for 

20 example. Also, F5(A/B) expresses the enhancement amount of the sharp 
signal, for example. These show processing equivalent to transforming 
the value A of the image signal and the value B of the processed signal 
into logarithmic space and then enhancing the respective differences in 
each, performing enhancement processing that is suited for the visual 

25 characteristics. 

With the visual processing device of this invention, it is possible 
to perform dynamic range compression where necessary while enhancing 
local contrast. 

The visual processing device according to attachment 9 is the 
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visual processing device according to attachment 8 in which the dynamic 
range compression function F4 is a direct proportion function of a 
proportion coefficient 1. 

With the visual processing device of this invention, it is possible 
5 to enhance the contrast of the image signal uniformly over a range 
spanning dark areas to bright areas. This contrast enhancement is 
enhancement processing that is suited for the visual characteristics. 

The visual processing device according to attachment 10 is the 
visual processing device according to attachment 8 in which the dynamic 
10 range compression function F4 is a monotonically increasing function. 

With the visual processing device of this invention, it is possible 
to use the dynamic range compression function F4, which is a 
monotonically increasing function, to perform dynamic range 
compression while enhancing local contrast. 
15 The visual processing device according to attachment 11 is the 

visual processing device according to attachment 10 in which the 
dynamic range compression function F4 is a concave function. 

With the visual processing device of this invention, it is possible 
to use the dynamic range compression function F4, which is a concave 
20 function, to perform dynamic range compression while enhancing local 
contrast. 

The visual processing device according to attachment 12 is the 
visual processing device according to attachment 8 in which the dynamic 
range compression function F4 is a power function. 
25 With the visual processing device of this invention, it is possible 

to use the dynamic range compression function F4, which is a power 
function, to transform the dynamic range while enhancing local contrast. 

The visual processing device according to attachment 13 is the 
visual processing device according to attachment 12 in which the 
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exponent of the power function in the dynamic range compression 
function F4 is determined based on a target contrast value, which is a 
target value of the contrast when displaying the image, and an actual 
contrast value, which is a contrast value in the display environment 
5 when displaying the image. 

Here, the target contrast value is the target value of the contrast 
when the image is displayed, and for example is a value that is 
determined by the dynamic range of the display device that will display 
the image. The actual contrast value is the contrast value in the 
10 display environment in which the image is to be displayed, and for 
example is a value that is determined by the contrast of the image 
displayed by the display device in a case where there is ambient light. 

With the visual processing device of this invention, through the 
dynamic range compression function F4 it is possible to compress the 
15 dynamic range of an image signal having a dynamic range that is equal 
to the target contrast value to a dynamic range that is equal to the 
actual contrast value. 

The visual processing device according to attachment 14 is the 
visual processing device according to any one of the attachments 8 to 13, 
20 in which the enhancement function F5 is a power function. 

With the visual processing device of this invention, it is possible 
to use the enhancement function F5, which is a power function, to 
enhance local contrast, allowing the dynamic range to be transformed 
visually. 

25 The visual processing device according to attachment 15 is the 

visual processing device according to attachment 14 in which the 
exponent of the power function in the enhancement function F5 is 
determined based on a target contrast value, which is a target value of 
the contrast when displaying the image, and an actual contrast value, 
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which is a contrast value in the display environment when displaying 
the image. 

With the visual processing device of this invention, it is possible 
to use the enhancement function F5, which is a power function, to 
5 enhance local contrast, allowing the dynamic range to be transformed 
visually. 

The visual processing device according to attachment 16 is the 
visual processing device according to attachment 14 or 15 in which the 
exponent of the power function in the enhancement function F5 is a 
10 value that decreases monotonically with respect to the value A of the 
image signal in a case where the value A of the image signal is larger 
than the value B of the processed signal. 

With the visual processing device of this invention, of those pixels 
being processed having a higher luminance than the surrounding pixels 
15 in the image signal, it is possible to lower the enhancement of local 
contrast in high-luminance areas. Thus, it is possible to inhibit 
"blooming" in the visually processed image. 

The visual processing device according to attachment 17 is the 
visual processing device according to attachment 14 or 15 in which the 
20 exponent of the power function in the enhancement function F5 is a 
value that increase monotonically with respect to the value A of the 
image signal in a case where the value A of the image signal is smaller 
than the value B of the processed signal. 

With the visual processing device of this invention, of those pixels 
25 being processed having a lower luminance than the surrounding pixels in 
the image signal, it is possible to lower the enhancement of local contrast 
in low-luminance areas. Thus, it is possible to inhibit "black squashing" 
in the visually processed image. 

The visual processing device according to attachment 18 is the 
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visual processing device according to attachment 14 or 15 in which the 
exponent of the power function in the enhancement function F5 is a 
value that increases monotonically with respect to the value A of the 
image signal in a case where the value A of the image signal is larger 
5 than the value B of the processed signal. 

With the visual processing device of this invention, of those pixels 
being processed having a higher luminance than the surrounding pixels 
in the image signal, it is possible to lower the enhancement of local 
contrast in low-luminance areas. Thus, the SN ratio is kept from 
10 becoming worse in the visually processed image. 

The visual processing device according to attachment 19 is the 
visual processing device according to attachment 14 or 15 in which the 
exponent of the power function in the enhancement function F5 is a 
value that increases monotonically with respect to the absolute value of 
15 the difference between the value A of the image signal and the value B of 
the processed signal. 

Here, it is also possible to define the value that monotonically 
increases with respect to the absolute value of the difference between the 
value A of the image signal and the value B of the processed signal as a 
20 value that is greater in magnitude the closer the ratio of the value A of 
the image signal to the value B of the processed signal is to one. 

With the visual processing device of this invention, it is possible 
to in particular enhance the local contrast of target pixels having little 
difference in brightness compared to the surrounding pixels in the image 
25 signal, and to keep from over enhancing the local contrast in pixels being 
processed having significant difference in brightness compared to the 
surrounding pixels in the image signal. 

The visual processing device according to attachment 20 is the 
visual processing device according to any one of attachments 14 to 19 in 
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which at least one of a maximum value or a minimum value of the 
enhancement function F5 is kept within a predetermined range. 

With the visual processing device of this invention, it is possible 
to keep the enhancement amount of the local contrast within an 
5 appropriate range. 

The visual processing device according to attachment 21 is the 
visual processing device according to attachment 8 in which the signal 
computation means has enhancement processing means and output 
processing means. The enhancement processing means performs 
10 enhancement processing of a divided signal that is obtained by dividing 
the image signal by the processed signal. The output processing means 
outputs the output signal based on the image signal and the divided 
signal that has been subjected to enhancement processing. 

With the visual processing device of this invention, enhancement 
15 processing means uses the enhancement function F5 to perform 
enhancement processing of a divided signal that is obtained by dividing 
the image signal by the processed signal. The output processing means 
outputs the output signal based on the image signal and the divided 
signal. 

20 The visual processing device according to attachment 22 is the 

visual processing device according to attachment 21 in which the output 
processing means obtains the product of the image signal and the divided 
signal that has been subjected to enhancement processing. 

With the visual processing device of this invention, the dynamic 

25 range compression function F4 is for example a direct proportion 
function of a proportion coefficient 1. 

The visual processing device according to attachment 23 is the 
visual processing device according to attachment 21 in which the output 
processing means includes DR compression means that performs 
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dynamic range (DR) compression with respect to the image signal, and 
obtains the product of the image signal after DR compression and the 
enhanced divided signal. 

With the visual processing device of this invention, the DR 
5 compression means performs dynamic range compression of the image 
signal using the dynamic range compression function F4. 

The visual processing device according to attachment 24 is the 
visual processing device according to any one of attachments 8 to 23, 
further comprising first transformation means and second 

10 transformation means. The first transformation means transforms 
input image data of a first predetermined range to a second 
predetermined range, and outputs this as the image signal. The second 
transformation means transforms the output signal of a third 
predetermined range to a fourth predetermined range, and outputs this 

15 as output image data. The second predetermined range is determined 
based on the target contrast value, which is the target value of the 
contrast when the image is displayed. The third predetermined range is 
determined based on the actual contrast value, which is the contrast 
value in the display environment when the image is displayed. 

20 With the visual processing device of this invention, it is possible 

to compress the dynamic range of the image overall down to the actual 
contrast value that has dropped due to the presence of ambient light, 
while locally maintaining the target contrast value. Thus, the image 
after visual processing has an improved visual effect. 

25 The visual processing device according to attachment 25 is the 

visual processing device according to attachment 24 in which the 
dynamic range compression function F4 is a function that transforms the 
image signal of the second predetermined range into the output signal of 
the third predetermined range. 
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With the visual processing device of this invention, the dynamic 
range of the image overall is compressed down to the third 
predetermined range due to the dynamic range compression function F4. 

The visual processing device according to attachment 26 is the 
5 visual processing device according to attachment 24 or 25 in which the 
first transformation means transforms the minimum value and the 
maximum value of the first predetermined range into the minimum 
value and the maximum value, respectively, of the second predetermined 
range. The second transformation means transforms the minimum 
10 value and the maximum value of the third predetermined range into the 
minimum value and the maximum value, respectively, of the fourth 
predetermined range. 

The visual processing device according to attachment 27 is the 
visual processing device according to attachment 26 in which the 
15 transformations by the first transformation means and the second 
transformation means are linear transformations. 

The visual processing device according to attachment 28 is the 
visual processing device according to any one of attachments 24 to 27, 
further comprising setting means that sets the third predetermined 
20 range. 

With the visual processing device of this invention, it is possible 
to set the third predetermined range in accordance with the display 
environment of the display device that is to display the image. Thus, it 
is possible to more suitably correct the ambient light. 
25 The visual processing device according to attachment 29 is the 

visual processing device according to attachment 28 in which the setting 
means includes storage means that stores a dynamic range of the display 
device for displaying an image, and measuring means that measures the 
luminance of the ambient light in the display environment when 
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displaying an image. 

With the visual processing device of this invention, it is possible 
to measure the luminance of the ambient light and then determine the 
actual contrast value from the luminance that has been measured and 
5 the dynamic range of the display device. 

The visual processing device according to attachment 30 is the 
visual processing device according to attachment 28 in which the setting 
means includes measuring means that measures the luminance in the 
display environment of the display device for displaying an image when 
10 displaying black level and when displaying white level. 

With the visual processing device of this invention, it is possible 
to measure the luminance when displaying black level and when 
displaying white level in the display environment and from these to 
determine the actual contrast value. 
15 The visual processing device according to attachment 31 

comprises input signal processing means and signal computation means. 
The input signal processing means performs spatial processing of an 
image signal that has been received as input, and outputs a processed 
signal. The signal computation means outputs an output signal based 
20 on a computation for enhancing a difference between the image signal 
and the processed signal in accordance with a value of the image signal. 

With the visual processing device of this invention, it is for 
example possible to enhance the sharp component of the image signal, 
which is the difference between the image signal and the processed 
25 signal, in accordance with the value of the image signal. Thus, it is 
possible to perform appropriate enhancement over a range spanning 
dark areas to bright areas of the image signal. 

The visual processing device according to attachment 32 is the 
visual processing device according to attachment 31 in which the signal 
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computation means outputs the output signal based on a computation 
that adds a value that is obtained by compressing the dynamic range of 
the image signal to the enhanced value from the computation for 
enhancing. 

5 With the visual processing device of this invention, it is for 

example possible to enhance the sharp component, for example, of the 
image signal in accordance with the value of the image signal and at the 
same time to compress the dynamic range. 

The visual processing device according to attachment 33 is the 

10 visual processing device according to attachment 31 or 32 in which the 
signal computation means computes a value C of the output signal with 
respect to a value A of the image signal, a value B of the processed signal, 
an enhancement amount adjustment function F6, an enhancement 
function F7, and a dynamic range compression function F8, according to 

15 the equation F8(A) + F6(A) * F7(A-B). 

Here, the value C of the output signal shows the following. That 
is, the difference (A-B) between the value A of the image signal and the 
value B of the processed signal expresses the sharp signal, for example. 
Also, F7(A-B) for example expresses the enhancement amount of the 

20 sharp signal. The enhancement amount is adjusted by the 
enhancement amount adjustment function F6 in accordance with the 
value A of the image signal, and where necessary is added to the value of 
the image signal after its dynamic range has been compressed. 

With the visual processing device of this invention, in areas 

25 where the value A of the image signal is large it is for example possible 
to maintain the contrast over a range spanning dark areas to bright 
areas by reducing the enhancement amount, for example. It is possible 
to maintain local contrast over a range spanning dark areas to bright 
areas even when dynamic range compression has been performed. 
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The visual processing device according to attachment 34 is the 
visual processing device according to attachment 33 in which the 
dynamic range compression function F8 is a direct proportion function of 
a proportion coefficient 1. 
5 With the visual processing device of this invention, it is possible 

to enhance the contrast uniformly over dark to bright areas of the image 
signal. 

The visual processing device according to attachment 35 is the 
visual processing device according to attachment 33 in which the 
10 dynamic range compression function F8 is a monotonically increasing 
function. 

With the visual processing device of this invention, it is possible 
to use the dynamic range compression function F8, which is a 
monotonically increasing function, to perform dynamic range 
15 compression while maintaining local contrast. 

The visual processing device according to attachment 36 is the 
visual processing device according to attachment 35 in which the 
dynamic range compression function F8 is a concave function. 

With the visual processing device of this invention, it is possible 
20 to use the dynamic range compression function F8, which is a concave 
function, to perform dynamic range compression while maintaining local 
contrast. 

The visual processing device according to attachment 37 is the 
visual processing device according to attachment 33 in which the 
25 dynamic range compression function F8 is a power function. 

With the visual processing device of this invention, it is possible 
to use the dynamic range compression function F8, which is a power 
function, to transform the dynamic range while maintaining local 
contrast. 
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The visual processing device according to attachment 38 is the 
visual processing device according to attachment 33 in which the signal 
computation means has enhancement processing means and output 
processing means. The enhancement processing means performs 
5 enhancement processing of a difference signal of the difference between 
the image signal and the processed signal in accordance with the pixel 
value of the image signal. The output processing means outputs the 
output signal based on the image signal and the enhanced difference 
signal. 

10 With the visual processing device of this invention, the 

enhancement processing means performs enhancement processing using 
the enhancement function F7, in which the enhancement amount has 
been adjusted by the enhancement amount adjustment function F6. 
The output processing means outputs an output signal based on the 

15 image signal and the difference signal. 

The visual processing device according to attachment 39 is the 
visual processing device according to attachment 38 in which the output 
processing means performs a summing process of taking the sum of the 
image signal and the enhanced difference signal. 

20 With the visual processing device of this invention, the dynamic 

range compression function F8 is for example a direct proportion 
function of a proportion coefficient 1. 

The visual processing device according to attachment 40 is the 
visual processing device according to attachment 38 in which the output 

25 processing means includes DR compression means that performs 
dynamic range (DR) compression with respect to the image signal, and 
performs a summing process of taking the sum of the DR compressed 
image signal and the enhanced difference signal. 

With the visual processing device of this invention, the DR 
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compression means uses the dynamic range compression function F8 to 
perform dynamic compression of the image signal. 

The visual processing device according to attachment 41 
comprises input signal processing means and signal computation means. 
5 The input signal processing means performs spatial processing of an 
image signal that has been received as input, and outputs a processed 
signal. The signal computation means outputs an output signal based 
on a computation that adds a value obtained by correcting the gradation 
of the image signal to the value obtained by enhancing the difference 

10 between the image signal and the processed signal. 

With the visual processing device of this invention, the difference 
between the image signal and the processed signal expresses the sharp 
component of the image signal. Also, enhancement of the sharp 
component and gradation correction of the image signal are performed 

15 independently. Thus, it is possible to perform a constant enhancement 
of the sharp component regardless of the gradation correction amount of 
the image signal. 

The visual processing device according to attachment 42 is the 
visual processing device according to attachment 41 in which the signal 

20 computation means computes a value C of the output signal with respect 
to a value A of the input signal, a value B of the processed signal, an 
enhancement function Fll, and a gradation correction function F12, 
according to the equation F12(A) + Fll(A-B). 

Here, the value C of the output signal shows the following. That 

25 is, the difference (A-B) between the value A of the image signal and the 
value B of the processed signal expresses the sharp signal, for example. 
Also, Fll(A-B) for example expresses the enhancement processing of the 
sharp signal. It also indicates that the gradation- corrected image signal 
and the enhanced sharp signal have been added. 
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With the visual processing device of this invention, it is possible 
to perform a fixed contrast enhancement regardless of the gradation 
correction. 

The visual processing device according to attachment 43 is the 
5 visual processing device according to attachment 42 in which the signal 
computation means has enhancement processing means and summing 
means. The enhancement processing means performs enhancement 
processing of a difference signal of the difference between the image 
signal and the processed signal. The summing means takes the sum of 

10 the gradation-corrected image signal and the enhanced difference signal 
and outputs the result as an output signal. 

With the visual processing device of this invention, the 
enhancement processing means performs enhancement processing of the 
difference signal using the enhancement function Fll. The summing 

15 means uses the gradation correction function F12 to take the sum of the 
gradation-processed image signal and the enhanced difference signal. 

The visual processing method according to attachment 44 
comprises a first transformation step, a signal computation step, and a 
second transformation step. The first transformation step is step of 

20 transforming an input image data of a first predetermined range to a 
second predetermined range, and taking this as an image signal. The 
signal computation step is a step of outputting an output signal of a 
third predetermined range based on a computation that includes at least 
one of a computation for compressing the dynamic range of the image 

25 signal, and a computation for enhancing a ratio between the image 
signal and a processed signal that is obtained by spatially processing the 
image signal. The second transformation step is a step of transforming 
the output signal of the third predetermined range to a fourth 
predetermined range, and taking this as output image data. The second 
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predetermined range is determined based on a target contrast value, 
which is a target value of the contrast when the image is displayed. 
The third predetermined range is determined based on an actual 
contrast value, which is a contrast value in the display environment 
5 when the image is displayed. 

With the visual processing method of this invention, it is for 
example possible to compress the dynamic range of the overall image 
down to the actual contrast value that has dropped due to the presence 
of ambient light, while locally maintaining the target contrast value. 

10 Thus, the image after visual processing has an improved visual effect. 

The visual processing device according to attachment 45 
comprises first transformation means, signal computation means, and 
second transformation means. The first transformation means 
transforms input image data of a first predetermined range to a second 

15 predetermined range, and takes this as an image signal. The signal 
computation means outputs an output signal of a third predetermined 
range based on a computation that includes at least one of a computation 
for compressing the dynamic range of the image signal, and a 
computation for enhancing a ratio between the image signal and a 

20 processed signal that is obtained by spatially processing the image signal. 
The second transformation means transforms the output signal of the 
third predetermined range to a fourth predetermined range, and takes 
this as output image data. The second predetermined range is 
determined based on a target contrast value, which is a target value of 

25 the contrast when the image is displayed. The third predetermined 
range is determined based on an actual contrast value, which is a 
contrast value in the display environment when the image is displayed. 

With the visual processing device of this invention, it is for 
example possible to compress the dynamic range of the overall image 
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down to the actual contrast value that has dropped due to the presence 
of ambient light, while locally maintaining the target contrast value. 
Thus, the image after visual processing has an improved visual effect. 

The visual processing program according to attachment 46 is a 
5 visual processing program for causing a computer to perform visual 
processing, the visual processing program causing the computer to 
perform a visual processing method that comprises a first transformation 
step, a signal computation step, and a second transformation step. 

The first transformation step is a step of transforming an input 

10 image data of a first predetermined range to a second predetermined 
range, and taking this as an image signal. The signal computation step 
is a step of outputting an output signal of a third predetermined range 
based on a computation that includes at least one of a computation for 
compressing the dynamic range of the image signal, and a computation 

15 for enhancing a ratio between the image signal and a processed signal 
that is obtained by spatially processing the image signal. The second 
transformation step is a step of transforming the output signal of the 
third predetermined range to a fourth predetermined range, and taking 
this as output image data. The second predetermined range is 

20 determined based on a target contrast value, which is a target value of 
the contrast when the image is displayed. The third predetermined 
range is determined based on an actual contrast value, which is a 
contrast value in the display environment when the image is displayed. 

With the visual processing program of this invention, it is for 

25 example possible to compress the dynamic range of the overall image 
down to the actual contrast value that has dropped due to the presence 
of ambient light, while locally maintaining the target contrast value. 
Thus, the image after visual processing has an improved visual effect. 

With the visual processing device of the present invention, it is 
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possible to obtain an image that has a better visual effect for the user 
who will view the visually processed image, and this device can be 
favorably employed as a visual processing device, and in particular, as a 
visual processing device that performs visual processing such as spatial 
5 processing or gradation processing of an image signal. 
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